{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "01afaac8",
   "metadata": {},
   "source": [
    "# Using eval_metric in XGBoost \n",
    "\n",
    "This notebook shows how to use XGBoost's Evaluation Metric (`eval_metric`) with H2O. \n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fcd3edb3",
   "metadata": {},
   "source": [
    "### What is eval_metric?\n",
    "\n",
    "Original (non-H2O) XGBoost library allows users to define one or more evaluation metrics that will be calculated on both training and validations datasets after each iteration. If no evaluation metric is defined by the user XGBoost will assign it based on the choice of the objective function. For example in the case of binary classification, XGBoost will use logloss to report performance on training and validation data. Logloss measures the extent to which predicted probabilities diverge from the actual class labels. You might want to consider evaluating the performance using a different metric, e.g. for imbalanced problems, where it is more important to correctly predict the positive minority class, you might want to use the Area under the PR curve and specify `eval_metric=\"aucpr\"`. Evaluation metric can easily be used for early stopping."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8b0e4ed4",
   "metadata": {},
   "source": [
    "### Is eval_metric needed with H2O XGBoost?\n",
    "\n",
    "H2O's approach to calculating metrics is different. By default, H2O will calculate all appropritate metrics for the given problem. For example for binary classification, H2O will report logloss, AUC, AUCPR and also additional metrics. Users thus typically don't need to worry about selecting the appropriate metric before model training. When early stopping is used, users will need to chose from one built-in early stopping metrics. For consistency between different model types and/or algorithm implementations, these are always calculated by H2O itself and, in XGBoost's specific case independect of XGBoost's eval_metric implementation."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9e6a1a1a",
   "metadata": {},
   "source": [
    "### When should eval_metric be considered?\n",
    "\n",
    "While you typically don't need to specify your custom `eval_metric`, there are cases doing so would be beneficial.\n",
    "\n",
    "Case 1: H2O doesn't provide a suitable built-in metric. Example: If you want to calculate classification error for a different threshold than the one automatically determined by H2O, you can do so only by specifying `eval_metric=\"error@<your threshold>\"` because H2O currently doesn't have this capability.\n",
    "\n",
    "Case 2: Frequent scoring. By default H2O uses timing-triggered scoring, it is trying to make sure that majority of the time is used on model training as opposed to just model scoring. You can override this behavior and specify manually at what iterations you want model to be scored (eg. each iteration, every 5th iteration, ...). Because H2O calculates all possible metrics as opposed just few in native XGBoost, and needs to extract XGBoost model from the native memory, it can have a significant overhead when user desires to score very frequently. This can slow down the model building."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09ce2240",
   "metadata": {},
   "source": [
    "### Example\n",
    "\n",
    "We will create a synthetic classification dataset, show XGBoost model training with default parameters and compare it to the output of model training with eval_metric used for early stopping."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "f18c59e7",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "versionFromGradle='3.39.0',projectVersion='3.39.0.99999',branch='michalk_eval-metric-ntb',lastCommitHash='256ed83c89220493cca0574ebd517eb09e3611fd',gitDescribe='jenkins-master-5998-dirty',compiledOn='2022-10-28 14:15:26',compiledBy='kurkami'\n",
      "Checking whether there is an H2O instance running at http://localhost:54321 . connected.\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "\n",
       "<style>\n",
       "\n",
       "#h2o-table-1.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-1 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-1 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-1 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-1 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-1 .h2o-table th,\n",
       "#h2o-table-1 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-1 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-1\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption></caption>\n",
       "    <thead></thead>\n",
       "    <tbody><tr><td>H2O_cluster_uptime:</td>\n",
       "<td>5 hours 14 mins</td></tr>\n",
       "<tr><td>H2O_cluster_timezone:</td>\n",
       "<td>America/New_York</td></tr>\n",
       "<tr><td>H2O_data_parsing_timezone:</td>\n",
       "<td>UTC</td></tr>\n",
       "<tr><td>H2O_cluster_version:</td>\n",
       "<td>3.39.0.99999</td></tr>\n",
       "<tr><td>H2O_cluster_version_age:</td>\n",
       "<td>1 day </td></tr>\n",
       "<tr><td>H2O_cluster_name:</td>\n",
       "<td>kurkami</td></tr>\n",
       "<tr><td>H2O_cluster_total_nodes:</td>\n",
       "<td>1</td></tr>\n",
       "<tr><td>H2O_cluster_free_memory:</td>\n",
       "<td>3.051 Gb</td></tr>\n",
       "<tr><td>H2O_cluster_total_cores:</td>\n",
       "<td>8</td></tr>\n",
       "<tr><td>H2O_cluster_allowed_cores:</td>\n",
       "<td>8</td></tr>\n",
       "<tr><td>H2O_cluster_status:</td>\n",
       "<td>locked, healthy</td></tr>\n",
       "<tr><td>H2O_connection_url:</td>\n",
       "<td>http://localhost:54321</td></tr>\n",
       "<tr><td>H2O_connection_proxy:</td>\n",
       "<td>{\"http\": null, \"https\": null}</td></tr>\n",
       "<tr><td>H2O_internal_security:</td>\n",
       "<td>False</td></tr>\n",
       "<tr><td>Python_version:</td>\n",
       "<td>3.10.6 final</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n"
      ],
      "text/plain": [
       "--------------------------  -----------------------------\n",
       "H2O_cluster_uptime:         5 hours 14 mins\n",
       "H2O_cluster_timezone:       America/New_York\n",
       "H2O_data_parsing_timezone:  UTC\n",
       "H2O_cluster_version:        3.39.0.99999\n",
       "H2O_cluster_version_age:    1 day\n",
       "H2O_cluster_name:           kurkami\n",
       "H2O_cluster_total_nodes:    1\n",
       "H2O_cluster_free_memory:    3.051 Gb\n",
       "H2O_cluster_total_cores:    8\n",
       "H2O_cluster_allowed_cores:  8\n",
       "H2O_cluster_status:         locked, healthy\n",
       "H2O_connection_url:         http://localhost:54321\n",
       "H2O_connection_proxy:       {\"http\": null, \"https\": null}\n",
       "H2O_internal_security:      False\n",
       "Python_version:             3.10.6 final\n",
       "--------------------------  -----------------------------"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "import h2o\n",
    "h2o.init(strict_version_check=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "9f3da16b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn.datasets import make_classification\n",
    "from sklearn.model_selection import train_test_split\n",
    "\n",
    "X, y = make_classification(n_samples=100000, \n",
    "                           n_informative=5,\n",
    "                           n_classes=2,\n",
    "                           random_state=42)\n",
    "X_train, X_validation, y_train, y_validation = train_test_split(X, y, test_size=0.25)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "172e6b47",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%\n",
      "Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%\n",
      "Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%\n",
      "Parse progress: |████████████████████████████████████████████████████████████████| (done) 100%\n"
     ]
    }
   ],
   "source": [
    "from h2o import H2OFrame\n",
    "train = H2OFrame(y_train, column_names=[\"y\"]).cbind(H2OFrame(X_train))\n",
    "valid = H2OFrame(y_validation, column_names=[\"y\"]).cbind(H2OFrame(X_validation))\n",
    "train[\"y\"] = train[\"y\"].asfactor()\n",
    "valid[\"y\"] = valid[\"y\"].asfactor()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "d08b576f",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<table class='dataframe'>\n",
       "<thead>\n",
       "<tr><th style=\"text-align: right;\">  y</th><th style=\"text-align: right;\">        C1</th><th style=\"text-align: right;\">       C2</th><th style=\"text-align: right;\">       C3</th><th style=\"text-align: right;\">        C4</th><th style=\"text-align: right;\">       C5</th><th style=\"text-align: right;\">        C6</th><th style=\"text-align: right;\">        C7</th><th style=\"text-align: right;\">       C8</th><th style=\"text-align: right;\">       C9</th><th style=\"text-align: right;\">       C10</th><th style=\"text-align: right;\">      C11</th><th style=\"text-align: right;\">      C12</th><th style=\"text-align: right;\">      C13</th><th style=\"text-align: right;\">       C14</th><th style=\"text-align: right;\">      C15</th><th style=\"text-align: right;\">       C16</th><th style=\"text-align: right;\">      C17</th><th style=\"text-align: right;\">      C18</th><th style=\"text-align: right;\">      C19</th><th style=\"text-align: right;\">      C20</th></tr>\n",
       "</thead>\n",
       "<tbody>\n",
       "<tr><td style=\"text-align: right;\">  1</td><td style=\"text-align: right;\">-0.460911 </td><td style=\"text-align: right;\"> 0.587072</td><td style=\"text-align: right;\"> 1.27653 </td><td style=\"text-align: right;\"> 1.65893  </td><td style=\"text-align: right;\"> 0.82732 </td><td style=\"text-align: right;\"> 0.900837 </td><td style=\"text-align: right;\"> 1.27825  </td><td style=\"text-align: right;\">-1.15342 </td><td style=\"text-align: right;\">-1.10419 </td><td style=\"text-align: right;\"> 0.231584 </td><td style=\"text-align: right;\"> 1.42246 </td><td style=\"text-align: right;\"> 0.632382</td><td style=\"text-align: right;\"> 1.6532  </td><td style=\"text-align: right;\"> 0.0807537</td><td style=\"text-align: right;\"> 0.63636 </td><td style=\"text-align: right;\">-0.0407703</td><td style=\"text-align: right;\">-0.712485</td><td style=\"text-align: right;\">-0.987504</td><td style=\"text-align: right;\"> 0.529023</td><td style=\"text-align: right;\"> 0.813985</td></tr>\n",
       "<tr><td style=\"text-align: right;\">  0</td><td style=\"text-align: right;\">-0.122517 </td><td style=\"text-align: right;\"> 0.926967</td><td style=\"text-align: right;\">-3.27666 </td><td style=\"text-align: right;\"> 1.65648  </td><td style=\"text-align: right;\"> 1.26381 </td><td style=\"text-align: right;\"> 0.866601 </td><td style=\"text-align: right;\">-0.112373 </td><td style=\"text-align: right;\"> 4.88467 </td><td style=\"text-align: right;\">-2.56306 </td><td style=\"text-align: right;\"> 1.37596  </td><td style=\"text-align: right;\"> 1.66414 </td><td style=\"text-align: right;\"> 0.750439</td><td style=\"text-align: right;\">-0.712589</td><td style=\"text-align: right;\"> 0.011264 </td><td style=\"text-align: right;\"> 0.338901</td><td style=\"text-align: right;\">-0.91639  </td><td style=\"text-align: right;\">-1.37998 </td><td style=\"text-align: right;\">-1.37817 </td><td style=\"text-align: right;\">-0.702391</td><td style=\"text-align: right;\">-0.222837</td></tr>\n",
       "<tr><td style=\"text-align: right;\">  1</td><td style=\"text-align: right;\"> 1.07287  </td><td style=\"text-align: right;\"> 1.35581 </td><td style=\"text-align: right;\"> 0.504084</td><td style=\"text-align: right;\">-1.9799   </td><td style=\"text-align: right;\"> 0.860703</td><td style=\"text-align: right;\">-0.753541 </td><td style=\"text-align: right;\"> 1.17781  </td><td style=\"text-align: right;\"> 0.364174</td><td style=\"text-align: right;\"> 2.49498 </td><td style=\"text-align: right;\">-0.309902 </td><td style=\"text-align: right;\"> 0.52471 </td><td style=\"text-align: right;\">-0.128492</td><td style=\"text-align: right;\"> 0.133623</td><td style=\"text-align: right;\"> 1.77568  </td><td style=\"text-align: right;\">-1.70823 </td><td style=\"text-align: right;\"> 0.263603 </td><td style=\"text-align: right;\">-1.69966 </td><td style=\"text-align: right;\"> 0.77875 </td><td style=\"text-align: right;\">-0.578858</td><td style=\"text-align: right;\"> 1.53087 </td></tr>\n",
       "<tr><td style=\"text-align: right;\">  1</td><td style=\"text-align: right;\">-0.497834 </td><td style=\"text-align: right;\"> 0.427476</td><td style=\"text-align: right;\">-1.67148 </td><td style=\"text-align: right;\">-1.71013  </td><td style=\"text-align: right;\"> 1.11473 </td><td style=\"text-align: right;\"> 0.0850045</td><td style=\"text-align: right;\">-2.81089  </td><td style=\"text-align: right;\"> 3.2892  </td><td style=\"text-align: right;\"> 0.676264</td><td style=\"text-align: right;\"> 1.04835  </td><td style=\"text-align: right;\"> 1.46596 </td><td style=\"text-align: right;\"> 0.818714</td><td style=\"text-align: right;\">-1.03108 </td><td style=\"text-align: right;\">-1.65528  </td><td style=\"text-align: right;\"> 0.301107</td><td style=\"text-align: right;\"> 1.19965  </td><td style=\"text-align: right;\"> 0.120588</td><td style=\"text-align: right;\"> 0.820176</td><td style=\"text-align: right;\">-0.930966</td><td style=\"text-align: right;\">-0.82086 </td></tr>\n",
       "<tr><td style=\"text-align: right;\">  1</td><td style=\"text-align: right;\"> 0.923038 </td><td style=\"text-align: right;\"> 1.51161 </td><td style=\"text-align: right;\"> 1.66059 </td><td style=\"text-align: right;\"> 1.45418  </td><td style=\"text-align: right;\">-2.3419  </td><td style=\"text-align: right;\">-0.757573 </td><td style=\"text-align: right;\"> 0.308842 </td><td style=\"text-align: right;\">-3.57087 </td><td style=\"text-align: right;\">-0.939606</td><td style=\"text-align: right;\"> 0.369506 </td><td style=\"text-align: right;\"> 0.990806</td><td style=\"text-align: right;\"> 0.67289 </td><td style=\"text-align: right;\"> 1.53918 </td><td style=\"text-align: right;\"> 1.79014  </td><td style=\"text-align: right;\"> 0.726961</td><td style=\"text-align: right;\"> 0.0339329</td><td style=\"text-align: right;\">-0.542889</td><td style=\"text-align: right;\">-1.01856 </td><td style=\"text-align: right;\"> 1.97659 </td><td style=\"text-align: right;\"> 1.50693 </td></tr>\n",
       "<tr><td style=\"text-align: right;\">  0</td><td style=\"text-align: right;\"> 0.575923 </td><td style=\"text-align: right;\"> 2.20724 </td><td style=\"text-align: right;\"> 0.595904</td><td style=\"text-align: right;\">-0.416675 </td><td style=\"text-align: right;\"> 0.21922 </td><td style=\"text-align: right;\"> 0.787082 </td><td style=\"text-align: right;\">-0.0753548</td><td style=\"text-align: right;\">-0.413841</td><td style=\"text-align: right;\">-0.280823</td><td style=\"text-align: right;\"> 0.756811 </td><td style=\"text-align: right;\">-0.115625</td><td style=\"text-align: right;\">-1.03631 </td><td style=\"text-align: right;\"> 0.617746</td><td style=\"text-align: right;\"> 1.15633  </td><td style=\"text-align: right;\">-1.3063  </td><td style=\"text-align: right;\"> 1.83909  </td><td style=\"text-align: right;\"> 0.610228</td><td style=\"text-align: right;\"> 0.62274 </td><td style=\"text-align: right;\">-0.157638</td><td style=\"text-align: right;\">-1.19873 </td></tr>\n",
       "<tr><td style=\"text-align: right;\">  0</td><td style=\"text-align: right;\">-0.931317 </td><td style=\"text-align: right;\"> 3.38117 </td><td style=\"text-align: right;\">-1.25809 </td><td style=\"text-align: right;\"> 0.323422 </td><td style=\"text-align: right;\"> 4.46615 </td><td style=\"text-align: right;\"> 0.366347 </td><td style=\"text-align: right;\">-0.178002 </td><td style=\"text-align: right;\"> 4.59046 </td><td style=\"text-align: right;\">-0.995683</td><td style=\"text-align: right;\"> 0.248196 </td><td style=\"text-align: right;\">-3.7437  </td><td style=\"text-align: right;\"> 1.22383 </td><td style=\"text-align: right;\">-1.00021 </td><td style=\"text-align: right;\">-0.754449 </td><td style=\"text-align: right;\"> 0.573945</td><td style=\"text-align: right;\"> 1.94618  </td><td style=\"text-align: right;\">-2.16485 </td><td style=\"text-align: right;\">-0.285859</td><td style=\"text-align: right;\">-3.99979 </td><td style=\"text-align: right;\">-1.39355 </td></tr>\n",
       "<tr><td style=\"text-align: right;\">  0</td><td style=\"text-align: right;\"> 0.956366 </td><td style=\"text-align: right;\"> 1.02616 </td><td style=\"text-align: right;\">-0.55144 </td><td style=\"text-align: right;\">-0.66649  </td><td style=\"text-align: right;\"> 2.47247 </td><td style=\"text-align: right;\"> 1.14188  </td><td style=\"text-align: right;\"> 1.10357  </td><td style=\"text-align: right;\"> 2.61941 </td><td style=\"text-align: right;\"> 1.2818  </td><td style=\"text-align: right;\"> 0.528047 </td><td style=\"text-align: right;\">-1.10656 </td><td style=\"text-align: right;\">-0.664716</td><td style=\"text-align: right;\">-0.849829</td><td style=\"text-align: right;\"> 1.49331  </td><td style=\"text-align: right;\">-0.183118</td><td style=\"text-align: right;\"> 0.618475 </td><td style=\"text-align: right;\">-0.413351</td><td style=\"text-align: right;\"> 0.410573</td><td style=\"text-align: right;\">-2.14432 </td><td style=\"text-align: right;\"> 2.35245 </td></tr>\n",
       "<tr><td style=\"text-align: right;\">  0</td><td style=\"text-align: right;\">-0.0523249</td><td style=\"text-align: right;\"> 1.73885 </td><td style=\"text-align: right;\">-1.47901 </td><td style=\"text-align: right;\"> 0.254941 </td><td style=\"text-align: right;\">-1.37106 </td><td style=\"text-align: right;\">-0.0549625</td><td style=\"text-align: right;\">-1.19023  </td><td style=\"text-align: right;\"> 0.848204</td><td style=\"text-align: right;\">-0.424188</td><td style=\"text-align: right;\">-0.0600663</td><td style=\"text-align: right;\">-0.781962</td><td style=\"text-align: right;\"> 1.1958  </td><td style=\"text-align: right;\">-0.456581</td><td style=\"text-align: right;\">-0.740832 </td><td style=\"text-align: right;\"> 2.26734 </td><td style=\"text-align: right;\"> 2.01902  </td><td style=\"text-align: right;\"> 1.89101 </td><td style=\"text-align: right;\"> 0.348762</td><td style=\"text-align: right;\"> 0.260157</td><td style=\"text-align: right;\"> 0.133197</td></tr>\n",
       "<tr><td style=\"text-align: right;\">  0</td><td style=\"text-align: right;\">-0.818169 </td><td style=\"text-align: right;\">-0.746954</td><td style=\"text-align: right;\">-1.28428 </td><td style=\"text-align: right;\">-0.0786964</td><td style=\"text-align: right;\"> 0.708029</td><td style=\"text-align: right;\">-0.252285 </td><td style=\"text-align: right;\">-0.19201  </td><td style=\"text-align: right;\"> 1.33074 </td><td style=\"text-align: right;\"> 0.229397</td><td style=\"text-align: right;\">-0.599344 </td><td style=\"text-align: right;\">-0.141543</td><td style=\"text-align: right;\">-0.721745</td><td style=\"text-align: right;\"> 0.442618</td><td style=\"text-align: right;\"> 0.953017 </td><td style=\"text-align: right;\"> 1.00111 </td><td style=\"text-align: right;\">-0.862009 </td><td style=\"text-align: right;\"> 0.790705</td><td style=\"text-align: right;\"> 1.56841 </td><td style=\"text-align: right;\"> 0.101824</td><td style=\"text-align: right;\"> 1.48008 </td></tr>\n",
       "</tbody>\n",
       "</table><pre style='font-size: smaller; margin-bottom: 1em;'>[75000 rows x 21 columns]</pre>"
      ],
      "text/plain": [
       "  y          C1         C2         C3          C4         C5          C6          C7         C8         C9         C10        C11        C12        C13         C14        C15         C16        C17        C18        C19        C20\n",
       "---  ----------  ---------  ---------  ----------  ---------  ----------  ----------  ---------  ---------  ----------  ---------  ---------  ---------  ----------  ---------  ----------  ---------  ---------  ---------  ---------\n",
       "  1  -0.460911    0.587072   1.27653    1.65893     0.82732    0.900837    1.27825    -1.15342   -1.10419    0.231584    1.42246    0.632382   1.6532     0.0807537   0.63636   -0.0407703  -0.712485  -0.987504   0.529023   0.813985\n",
       "  0  -0.122517    0.926967  -3.27666    1.65648     1.26381    0.866601   -0.112373    4.88467   -2.56306    1.37596     1.66414    0.750439  -0.712589   0.011264    0.338901  -0.91639    -1.37998   -1.37817   -0.702391  -0.222837\n",
       "  1   1.07287     1.35581    0.504084  -1.9799      0.860703  -0.753541    1.17781     0.364174   2.49498   -0.309902    0.52471   -0.128492   0.133623   1.77568    -1.70823    0.263603   -1.69966    0.77875   -0.578858   1.53087\n",
       "  1  -0.497834    0.427476  -1.67148   -1.71013     1.11473    0.0850045  -2.81089     3.2892     0.676264   1.04835     1.46596    0.818714  -1.03108   -1.65528     0.301107   1.19965     0.120588   0.820176  -0.930966  -0.82086\n",
       "  1   0.923038    1.51161    1.66059    1.45418    -2.3419    -0.757573    0.308842   -3.57087   -0.939606   0.369506    0.990806   0.67289    1.53918    1.79014     0.726961   0.0339329  -0.542889  -1.01856    1.97659    1.50693\n",
       "  0   0.575923    2.20724    0.595904  -0.416675    0.21922    0.787082   -0.0753548  -0.413841  -0.280823   0.756811   -0.115625  -1.03631    0.617746   1.15633    -1.3063     1.83909     0.610228   0.62274   -0.157638  -1.19873\n",
       "  0  -0.931317    3.38117   -1.25809    0.323422    4.46615    0.366347   -0.178002    4.59046   -0.995683   0.248196   -3.7437     1.22383   -1.00021   -0.754449    0.573945   1.94618    -2.16485   -0.285859  -3.99979   -1.39355\n",
       "  0   0.956366    1.02616   -0.55144   -0.66649     2.47247    1.14188     1.10357     2.61941    1.2818     0.528047   -1.10656   -0.664716  -0.849829   1.49331    -0.183118   0.618475   -0.413351   0.410573  -2.14432    2.35245\n",
       "  0  -0.0523249   1.73885   -1.47901    0.254941   -1.37106   -0.0549625  -1.19023     0.848204  -0.424188  -0.0600663  -0.781962   1.1958    -0.456581  -0.740832    2.26734    2.01902     1.89101    0.348762   0.260157   0.133197\n",
       "  0  -0.818169   -0.746954  -1.28428   -0.0786964   0.708029  -0.252285   -0.19201     1.33074    0.229397  -0.599344   -0.141543  -0.721745   0.442618   0.953017    1.00111   -0.862009    0.790705   1.56841    0.101824   1.48008\n",
       "[75000 rows x 21 columns]\n"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "train"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "24ddbce8",
   "metadata": {},
   "source": [
    "##### Train XGBoost model with logloss as stopping metric"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f29a8bbe",
   "metadata": {},
   "source": [
    "We are specifying 1000 trees to be built and providing stopping criteria to make the model stop reasonably early for the purpose of this example."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "525ad2d1",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xgboost Model Build progress: |██████████████████████████████████████████████████| (done) 100%\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style='margin: 1em 0 1em 0;'>Model Details\n",
       "=============\n",
       "H2OXGBoostEstimator : XGBoost\n",
       "Model Key: XGBoost_model_python_1666962310297_518\n",
       "</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-2.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-2 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-2 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-2 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-2 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-2 .h2o-table th,\n",
       "#h2o-table-2 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-2 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-2\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Model Summary: </caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>number_of_trees</th></tr></thead>\n",
       "    <tbody><tr><td></td>\n",
       "<td>22.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'><pre style='margin: 1em 0 1em 0;'>ModelMetricsBinomial: xgboost\n",
       "** Reported on train data. **\n",
       "\n",
       "MSE: 0.03475774769265435\n",
       "RMSE: 0.18643429859511995\n",
       "LogLoss: 0.12901406890397824\n",
       "Mean Per-Class Error: 0.04387881533186782\n",
       "AUC: 0.9893340459497513\n",
       "AUCPR: 0.988704085388505\n",
       "Gini: 0.9786680918995025</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-3.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-3 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-3 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-3 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-3 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-3 .h2o-table th,\n",
       "#h2o-table-3 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-3 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-3\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.5101676438305829</caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>0</th>\n",
       "<th>1</th>\n",
       "<th>Error</th>\n",
       "<th>Rate</th></tr></thead>\n",
       "    <tbody><tr><td>0</td>\n",
       "<td>35155.0</td>\n",
       "<td>2174.0</td>\n",
       "<td>0.0582</td>\n",
       "<td> (2174.0/37329.0)</td></tr>\n",
       "<tr><td>1</td>\n",
       "<td>1112.0</td>\n",
       "<td>36559.0</td>\n",
       "<td>0.0295</td>\n",
       "<td> (1112.0/37671.0)</td></tr>\n",
       "<tr><td>Total</td>\n",
       "<td>36267.0</td>\n",
       "<td>38733.0</td>\n",
       "<td>0.0438</td>\n",
       "<td> (3286.0/75000.0)</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-4.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-4 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-4 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-4 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-4 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-4 .h2o-table th,\n",
       "#h2o-table-4 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-4 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-4\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Maximum Metrics: Maximum metrics at their respective thresholds</caption>\n",
       "    <thead><tr><th>metric</th>\n",
       "<th>threshold</th>\n",
       "<th>value</th>\n",
       "<th>idx</th></tr></thead>\n",
       "    <tbody><tr><td>max f1</td>\n",
       "<td>0.5101676</td>\n",
       "<td>0.9569918</td>\n",
       "<td>214.0</td></tr>\n",
       "<tr><td>max f2</td>\n",
       "<td>0.3082806</td>\n",
       "<td>0.9715081</td>\n",
       "<td>263.0</td></tr>\n",
       "<tr><td>max f0point5</td>\n",
       "<td>0.7532607</td>\n",
       "<td>0.9592084</td>\n",
       "<td>147.0</td></tr>\n",
       "<tr><td>max accuracy</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9564</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max precision</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max recall</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max specificity</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max absolute_mcc</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9129277</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max min_per_class_accuracy</td>\n",
       "<td>0.6239853</td>\n",
       "<td>0.9548073</td>\n",
       "<td>184.0</td></tr>\n",
       "<tr><td>max mean_per_class_accuracy</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9563597</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max tns</td>\n",
       "<td>0.9974565</td>\n",
       "<td>37329.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fns</td>\n",
       "<td>0.9974565</td>\n",
       "<td>37460.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fps</td>\n",
       "<td>0.0016028</td>\n",
       "<td>37329.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tps</td>\n",
       "<td>0.0016028</td>\n",
       "<td>37671.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tnr</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fnr</td>\n",
       "<td>0.9974565</td>\n",
       "<td>0.9943989</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fpr</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tpr</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-5.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-5 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-5 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-5 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-5 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-5 .h2o-table th,\n",
       "#h2o-table-5 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-5 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-5\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Gains/Lift Table: Avg response rate: 50.23 %, avg score: 50.22 %</caption>\n",
       "    <thead><tr><th>group</th>\n",
       "<th>cumulative_data_fraction</th>\n",
       "<th>lower_threshold</th>\n",
       "<th>lift</th>\n",
       "<th>cumulative_lift</th>\n",
       "<th>response_rate</th>\n",
       "<th>score</th>\n",
       "<th>cumulative_response_rate</th>\n",
       "<th>cumulative_score</th>\n",
       "<th>capture_rate</th>\n",
       "<th>cumulative_capture_rate</th>\n",
       "<th>gain</th>\n",
       "<th>cumulative_gain</th>\n",
       "<th>kolmogorov_smirnov</th></tr></thead>\n",
       "    <tbody><tr><td>1</td>\n",
       "<td>0.0101733</td>\n",
       "<td>0.9961097</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967516</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967516</td>\n",
       "<td>0.0202543</td>\n",
       "<td>0.0202543</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0202543</td></tr>\n",
       "<tr><td>2</td>\n",
       "<td>0.0200267</td>\n",
       "<td>0.9957153</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9958894</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9963274</td>\n",
       "<td>0.0196172</td>\n",
       "<td>0.0398715</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0398715</td></tr>\n",
       "<tr><td>3</td>\n",
       "<td>0.03068</td>\n",
       "<td>0.9953964</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9955495</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9960573</td>\n",
       "<td>0.0212099</td>\n",
       "<td>0.0610815</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0610815</td></tr>\n",
       "<tr><td>4</td>\n",
       "<td>0.0401067</td>\n",
       "<td>0.9951944</td>\n",
       "<td>1.9881054</td>\n",
       "<td>1.9902595</td>\n",
       "<td>0.9985856</td>\n",
       "<td>0.9952708</td>\n",
       "<td>0.9996676</td>\n",
       "<td>0.9958724</td>\n",
       "<td>0.0187412</td>\n",
       "<td>0.0798227</td>\n",
       "<td>98.8105385</td>\n",
       "<td>99.0259523</td>\n",
       "<td>0.0797959</td></tr>\n",
       "<tr><td>5</td>\n",
       "<td>0.0517733</td>\n",
       "<td>0.9949297</td>\n",
       "<td>1.9863707</td>\n",
       "<td>1.9893832</td>\n",
       "<td>0.9977143</td>\n",
       "<td>0.9950084</td>\n",
       "<td>0.9992274</td>\n",
       "<td>0.9956777</td>\n",
       "<td>0.0231743</td>\n",
       "<td>0.1029970</td>\n",
       "<td>98.6370721</td>\n",
       "<td>98.9383216</td>\n",
       "<td>0.1029166</td></tr>\n",
       "<tr><td>6</td>\n",
       "<td>0.1000133</td>\n",
       "<td>0.9933097</td>\n",
       "<td>1.9881700</td>\n",
       "<td>1.9887980</td>\n",
       "<td>0.9986180</td>\n",
       "<td>0.9941657</td>\n",
       "<td>0.9989335</td>\n",
       "<td>0.9949484</td>\n",
       "<td>0.0959093</td>\n",
       "<td>0.1989063</td>\n",
       "<td>98.8169987</td>\n",
       "<td>98.8798032</td>\n",
       "<td>0.1986920</td></tr>\n",
       "<tr><td>7</td>\n",
       "<td>0.15008</td>\n",
       "<td>0.9904984</td>\n",
       "<td>1.9840287</td>\n",
       "<td>1.9872070</td>\n",
       "<td>0.9965379</td>\n",
       "<td>0.9920577</td>\n",
       "<td>0.9981343</td>\n",
       "<td>0.9939841</td>\n",
       "<td>0.0993337</td>\n",
       "<td>0.2982400</td>\n",
       "<td>98.4028728</td>\n",
       "<td>98.7206993</td>\n",
       "<td>0.2976775</td></tr>\n",
       "<tr><td>8</td>\n",
       "<td>0.2000133</td>\n",
       "<td>0.9849605</td>\n",
       "<td>1.9824155</td>\n",
       "<td>1.9860108</td>\n",
       "<td>0.9957276</td>\n",
       "<td>0.9879125</td>\n",
       "<td>0.9975335</td>\n",
       "<td>0.9924683</td>\n",
       "<td>0.0989886</td>\n",
       "<td>0.3972286</td>\n",
       "<td>98.2415459</td>\n",
       "<td>98.6010786</td>\n",
       "<td>0.3962375</td></tr>\n",
       "<tr><td>9</td>\n",
       "<td>0.30004</td>\n",
       "<td>0.9561046</td>\n",
       "<td>1.9673021</td>\n",
       "<td>1.9797737</td>\n",
       "<td>0.9881365</td>\n",
       "<td>0.9740387</td>\n",
       "<td>0.9944007</td>\n",
       "<td>0.9863243</td>\n",
       "<td>0.1967827</td>\n",
       "<td>0.5940113</td>\n",
       "<td>96.7302096</td>\n",
       "<td>97.9773725</td>\n",
       "<td>0.5906359</td></tr>\n",
       "<tr><td>10</td>\n",
       "<td>0.4000133</td>\n",
       "<td>0.8770613</td>\n",
       "<td>1.9123254</td>\n",
       "<td>1.9629167</td>\n",
       "<td>0.9605228</td>\n",
       "<td>0.9210076</td>\n",
       "<td>0.9859338</td>\n",
       "<td>0.9700000</td>\n",
       "<td>0.1911815</td>\n",
       "<td>0.7851929</td>\n",
       "<td>91.2325408</td>\n",
       "<td>96.2916704</td>\n",
       "<td>0.7738880</td></tr>\n",
       "<tr><td>11</td>\n",
       "<td>0.5000133</td>\n",
       "<td>0.6383476</td>\n",
       "<td>1.6763558</td>\n",
       "<td>1.9056061</td>\n",
       "<td>0.842</td>\n",
       "<td>0.8051186</td>\n",
       "<td>0.9571478</td>\n",
       "<td>0.9370246</td>\n",
       "<td>0.1676356</td>\n",
       "<td>0.9528284</td>\n",
       "<td>67.6355817</td>\n",
       "<td>90.5606055</td>\n",
       "<td>0.9097788</td></tr>\n",
       "<tr><td>12</td>\n",
       "<td>0.6</td>\n",
       "<td>0.0781245</td>\n",
       "<td>0.4261140</td>\n",
       "<td>1.6590569</td>\n",
       "<td>0.2140285</td>\n",
       "<td>0.2876185</td>\n",
       "<td>0.8333111</td>\n",
       "<td>0.8288047</td>\n",
       "<td>0.0426057</td>\n",
       "<td>0.9954342</td>\n",
       "<td>-57.3886006</td>\n",
       "<td>65.9056923</td>\n",
       "<td>0.7944912</td></tr>\n",
       "<tr><td>13</td>\n",
       "<td>0.7000133</td>\n",
       "<td>0.0128222</td>\n",
       "<td>0.0238879</td>\n",
       "<td>1.4254346</td>\n",
       "<td>0.0119984</td>\n",
       "<td>0.0339001</td>\n",
       "<td>0.7159673</td>\n",
       "<td>0.7152339</td>\n",
       "<td>0.0023891</td>\n",
       "<td>0.9978233</td>\n",
       "<td>-97.6112128</td>\n",
       "<td>42.5434648</td>\n",
       "<td>0.5983483</td></tr>\n",
       "<tr><td>14</td>\n",
       "<td>0.80008</td>\n",
       "<td>0.0057930</td>\n",
       "<td>0.0122028</td>\n",
       "<td>1.2486806</td>\n",
       "<td>0.0061292</td>\n",
       "<td>0.0084165</td>\n",
       "<td>0.6271873</td>\n",
       "<td>0.6268317</td>\n",
       "<td>0.0012211</td>\n",
       "<td>0.9990444</td>\n",
       "<td>-98.7797151</td>\n",
       "<td>24.8680579</td>\n",
       "<td>0.3997516</td></tr>\n",
       "<tr><td>15</td>\n",
       "<td>0.9</td>\n",
       "<td>0.0034428</td>\n",
       "<td>0.0045164</td>\n",
       "<td>1.1105507</td>\n",
       "<td>0.0022685</td>\n",
       "<td>0.0045299</td>\n",
       "<td>0.5578074</td>\n",
       "<td>0.5577424</td>\n",
       "<td>0.0004513</td>\n",
       "<td>0.9994956</td>\n",
       "<td>-99.5483632</td>\n",
       "<td>11.0550704</td>\n",
       "<td>0.1999028</td></tr>\n",
       "<tr><td>16</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0001672</td>\n",
       "<td>0.0050437</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0025333</td>\n",
       "<td>0.0026268</td>\n",
       "<td>0.50228</td>\n",
       "<td>0.5022308</td>\n",
       "<td>0.0005044</td>\n",
       "<td>1.0</td>\n",
       "<td>-99.4956332</td>\n",
       "<td>0.0</td>\n",
       "<td>0.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div></div>\n",
       "<div style='margin: 1em 0 1em 0;'><pre style='margin: 1em 0 1em 0;'>ModelMetricsBinomial: xgboost\n",
       "** Reported on validation data. **\n",
       "\n",
       "MSE: 0.038600168581279404\n",
       "RMSE: 0.1964692560714765\n",
       "LogLoss: 0.14264211397729937\n",
       "Mean Per-Class Error: 0.04874265748103474\n",
       "AUC: 0.9860367984758349\n",
       "AUCPR: 0.9841777019906762\n",
       "Gini: 0.9720735969516698</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-6.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-6 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-6 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-6 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-6 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-6 .h2o-table th,\n",
       "#h2o-table-6 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-6 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-6\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.4899955771186135</caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>0</th>\n",
       "<th>1</th>\n",
       "<th>Error</th>\n",
       "<th>Rate</th></tr></thead>\n",
       "    <tbody><tr><td>0</td>\n",
       "<td>11844.0</td>\n",
       "<td>840.0</td>\n",
       "<td>0.0662</td>\n",
       "<td> (840.0/12684.0)</td></tr>\n",
       "<tr><td>1</td>\n",
       "<td>385.0</td>\n",
       "<td>11931.0</td>\n",
       "<td>0.0313</td>\n",
       "<td> (385.0/12316.0)</td></tr>\n",
       "<tr><td>Total</td>\n",
       "<td>12229.0</td>\n",
       "<td>12771.0</td>\n",
       "<td>0.049</td>\n",
       "<td> (1225.0/25000.0)</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-7.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-7 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-7 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-7 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-7 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-7 .h2o-table th,\n",
       "#h2o-table-7 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-7 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-7\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Maximum Metrics: Maximum metrics at their respective thresholds</caption>\n",
       "    <thead><tr><th>metric</th>\n",
       "<th>threshold</th>\n",
       "<th>value</th>\n",
       "<th>idx</th></tr></thead>\n",
       "    <tbody><tr><td>max f1</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9511699</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max f2</td>\n",
       "<td>0.2381639</td>\n",
       "<td>0.9684011</td>\n",
       "<td>281.0</td></tr>\n",
       "<tr><td>max f0point5</td>\n",
       "<td>0.7637601</td>\n",
       "<td>0.9521032</td>\n",
       "<td>144.0</td></tr>\n",
       "<tr><td>max accuracy</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.951</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max precision</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max recall</td>\n",
       "<td>0.0018354</td>\n",
       "<td>1.0</td>\n",
       "<td>398.0</td></tr>\n",
       "<tr><td>max specificity</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max absolute_mcc</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9026291</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max min_per_class_accuracy</td>\n",
       "<td>0.6303638</td>\n",
       "<td>0.9494966</td>\n",
       "<td>182.0</td></tr>\n",
       "<tr><td>max mean_per_class_accuracy</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9512573</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max tns</td>\n",
       "<td>0.9974683</td>\n",
       "<td>12684.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fns</td>\n",
       "<td>0.9974683</td>\n",
       "<td>12244.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fps</td>\n",
       "<td>0.0013435</td>\n",
       "<td>12684.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tps</td>\n",
       "<td>0.0018354</td>\n",
       "<td>12316.0</td>\n",
       "<td>398.0</td></tr>\n",
       "<tr><td>max tnr</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fnr</td>\n",
       "<td>0.9974683</td>\n",
       "<td>0.9941539</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fpr</td>\n",
       "<td>0.0013435</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tpr</td>\n",
       "<td>0.0018354</td>\n",
       "<td>1.0</td>\n",
       "<td>398.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-8.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-8 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-8 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-8 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-8 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-8 .h2o-table th,\n",
       "#h2o-table-8 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-8 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-8\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Gains/Lift Table: Avg response rate: 49.26 %, avg score: 49.47 %</caption>\n",
       "    <thead><tr><th>group</th>\n",
       "<th>cumulative_data_fraction</th>\n",
       "<th>lower_threshold</th>\n",
       "<th>lift</th>\n",
       "<th>cumulative_lift</th>\n",
       "<th>response_rate</th>\n",
       "<th>score</th>\n",
       "<th>cumulative_response_rate</th>\n",
       "<th>cumulative_score</th>\n",
       "<th>capture_rate</th>\n",
       "<th>cumulative_capture_rate</th>\n",
       "<th>gain</th>\n",
       "<th>cumulative_gain</th>\n",
       "<th>kolmogorov_smirnov</th></tr></thead>\n",
       "    <tbody><tr><td>1</td>\n",
       "<td>0.01028</td>\n",
       "<td>0.9961244</td>\n",
       "<td>2.0298798</td>\n",
       "<td>2.0298798</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967313</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967313</td>\n",
       "<td>0.0208672</td>\n",
       "<td>0.0208672</td>\n",
       "<td>102.9879831</td>\n",
       "<td>102.9879831</td>\n",
       "<td>0.0208672</td></tr>\n",
       "<tr><td>2</td>\n",
       "<td>0.02032</td>\n",
       "<td>0.9958012</td>\n",
       "<td>2.0298798</td>\n",
       "<td>2.0298798</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9959175</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9963292</td>\n",
       "<td>0.0203800</td>\n",
       "<td>0.0412472</td>\n",
       "<td>102.9879831</td>\n",
       "<td>102.9879831</td>\n",
       "<td>0.0412472</td></tr>\n",
       "<tr><td>3</td>\n",
       "<td>0.03016</td>\n",
       "<td>0.9954615</td>\n",
       "<td>2.0216283</td>\n",
       "<td>2.0271877</td>\n",
       "<td>0.9959350</td>\n",
       "<td>0.9956102</td>\n",
       "<td>0.9986737</td>\n",
       "<td>0.9960946</td>\n",
       "<td>0.0198928</td>\n",
       "<td>0.0611400</td>\n",
       "<td>102.1628287</td>\n",
       "<td>102.7187683</td>\n",
       "<td>0.0610611</td></tr>\n",
       "<tr><td>4</td>\n",
       "<td>0.04108</td>\n",
       "<td>0.9951944</td>\n",
       "<td>2.0224444</td>\n",
       "<td>2.0259268</td>\n",
       "<td>0.9963370</td>\n",
       "<td>0.9952887</td>\n",
       "<td>0.9980526</td>\n",
       "<td>0.9958804</td>\n",
       "<td>0.0220851</td>\n",
       "<td>0.0832251</td>\n",
       "<td>102.2444374</td>\n",
       "<td>102.5926803</td>\n",
       "<td>0.0830674</td></tr>\n",
       "<tr><td>5</td>\n",
       "<td>0.05256</td>\n",
       "<td>0.9949297</td>\n",
       "<td>2.0157343</td>\n",
       "<td>2.0237006</td>\n",
       "<td>0.9930314</td>\n",
       "<td>0.9950085</td>\n",
       "<td>0.9969559</td>\n",
       "<td>0.9956900</td>\n",
       "<td>0.0231406</td>\n",
       "<td>0.1063657</td>\n",
       "<td>101.5734327</td>\n",
       "<td>102.3700593</td>\n",
       "<td>0.1060503</td></tr>\n",
       "<tr><td>6</td>\n",
       "<td>0.1</td>\n",
       "<td>0.9932859</td>\n",
       "<td>2.0196106</td>\n",
       "<td>2.0217603</td>\n",
       "<td>0.9949410</td>\n",
       "<td>0.9941552</td>\n",
       "<td>0.996</td>\n",
       "<td>0.9949619</td>\n",
       "<td>0.0958103</td>\n",
       "<td>0.2021760</td>\n",
       "<td>101.9610625</td>\n",
       "<td>102.1760312</td>\n",
       "<td>0.2013876</td></tr>\n",
       "<tr><td>7</td>\n",
       "<td>0.15</td>\n",
       "<td>0.9902694</td>\n",
       "<td>2.0233842</td>\n",
       "<td>2.0223016</td>\n",
       "<td>0.9968</td>\n",
       "<td>0.9919490</td>\n",
       "<td>0.9962667</td>\n",
       "<td>0.9939576</td>\n",
       "<td>0.1011692</td>\n",
       "<td>0.3033452</td>\n",
       "<td>102.3384216</td>\n",
       "<td>102.2301613</td>\n",
       "<td>0.3022415</td></tr>\n",
       "<tr><td>8</td>\n",
       "<td>0.2</td>\n",
       "<td>0.9840660</td>\n",
       "<td>2.0136408</td>\n",
       "<td>2.0201364</td>\n",
       "<td>0.992</td>\n",
       "<td>0.9873945</td>\n",
       "<td>0.9952</td>\n",
       "<td>0.9923168</td>\n",
       "<td>0.1006820</td>\n",
       "<td>0.4040273</td>\n",
       "<td>101.3640792</td>\n",
       "<td>102.0136408</td>\n",
       "<td>0.4021351</td></tr>\n",
       "<tr><td>9</td>\n",
       "<td>0.3</td>\n",
       "<td>0.9523897</td>\n",
       "<td>1.9852225</td>\n",
       "<td>2.0084984</td>\n",
       "<td>0.978</td>\n",
       "<td>0.9717051</td>\n",
       "<td>0.9894667</td>\n",
       "<td>0.9854462</td>\n",
       "<td>0.1985222</td>\n",
       "<td>0.6025495</td>\n",
       "<td>98.5222475</td>\n",
       "<td>100.8498430</td>\n",
       "<td>0.5963212</td></tr>\n",
       "<tr><td>10</td>\n",
       "<td>0.4</td>\n",
       "<td>0.8671057</td>\n",
       "<td>1.9235141</td>\n",
       "<td>1.9872524</td>\n",
       "<td>0.9476</td>\n",
       "<td>0.9142567</td>\n",
       "<td>0.979</td>\n",
       "<td>0.9676488</td>\n",
       "<td>0.1923514</td>\n",
       "<td>0.7949009</td>\n",
       "<td>92.3514128</td>\n",
       "<td>98.7252355</td>\n",
       "<td>0.7783447</td></tr>\n",
       "<tr><td>11</td>\n",
       "<td>0.5</td>\n",
       "<td>0.5822353</td>\n",
       "<td>1.6174082</td>\n",
       "<td>1.9132835</td>\n",
       "<td>0.7968</td>\n",
       "<td>0.7825059</td>\n",
       "<td>0.94256</td>\n",
       "<td>0.9306202</td>\n",
       "<td>0.1617408</td>\n",
       "<td>0.9566418</td>\n",
       "<td>61.7408249</td>\n",
       "<td>91.3283534</td>\n",
       "<td>0.9000350</td></tr>\n",
       "<tr><td>12</td>\n",
       "<td>0.6</td>\n",
       "<td>0.0667765</td>\n",
       "<td>0.3775576</td>\n",
       "<td>1.6573292</td>\n",
       "<td>0.186</td>\n",
       "<td>0.2480180</td>\n",
       "<td>0.8164667</td>\n",
       "<td>0.8168532</td>\n",
       "<td>0.0377558</td>\n",
       "<td>0.9943975</td>\n",
       "<td>-62.2442351</td>\n",
       "<td>65.7329219</td>\n",
       "<td>0.7773524</td></tr>\n",
       "<tr><td>13</td>\n",
       "<td>0.7</td>\n",
       "<td>0.0125771</td>\n",
       "<td>0.0186749</td>\n",
       "<td>1.4232357</td>\n",
       "<td>0.0092</td>\n",
       "<td>0.0304482</td>\n",
       "<td>0.7011429</td>\n",
       "<td>0.7045096</td>\n",
       "<td>0.0018675</td>\n",
       "<td>0.9962650</td>\n",
       "<td>-98.1325106</td>\n",
       "<td>42.3235744</td>\n",
       "<td>0.5839345</td></tr>\n",
       "<tr><td>14</td>\n",
       "<td>0.80048</td>\n",
       "<td>0.0057412</td>\n",
       "<td>0.0185857</td>\n",
       "<td>1.2469175</td>\n",
       "<td>0.0091561</td>\n",
       "<td>0.0081574</td>\n",
       "<td>0.6142814</td>\n",
       "<td>0.6171002</td>\n",
       "<td>0.0018675</td>\n",
       "<td>0.9981325</td>\n",
       "<td>-98.1414317</td>\n",
       "<td>24.6917488</td>\n",
       "<td>0.3895705</td></tr>\n",
       "<tr><td>15</td>\n",
       "<td>0.9014</td>\n",
       "<td>0.0034592</td>\n",
       "<td>0.0104592</td>\n",
       "<td>1.1084846</td>\n",
       "<td>0.0051526</td>\n",
       "<td>0.0045211</td>\n",
       "<td>0.5460839</td>\n",
       "<td>0.5485164</td>\n",
       "<td>0.0010555</td>\n",
       "<td>0.9991880</td>\n",
       "<td>-98.9540849</td>\n",
       "<td>10.8484633</td>\n",
       "<td>0.1927390</td></tr>\n",
       "<tr><td>16</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0002609</td>\n",
       "<td>0.0082348</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0040568</td>\n",
       "<td>0.0026446</td>\n",
       "<td>0.49264</td>\n",
       "<td>0.4946934</td>\n",
       "<td>0.0008120</td>\n",
       "<td>1.0</td>\n",
       "<td>-99.1765193</td>\n",
       "<td>0.0</td>\n",
       "<td>0.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div></div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-9.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-9 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-9 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-9 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-9 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-9 .h2o-table th,\n",
       "#h2o-table-9 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-9 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-9\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Scoring History: </caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>timestamp</th>\n",
       "<th>duration</th>\n",
       "<th>number_of_trees</th>\n",
       "<th>training_rmse</th>\n",
       "<th>training_logloss</th>\n",
       "<th>training_auc</th>\n",
       "<th>training_pr_auc</th>\n",
       "<th>training_lift</th>\n",
       "<th>training_classification_error</th>\n",
       "<th>validation_rmse</th>\n",
       "<th>validation_logloss</th>\n",
       "<th>validation_auc</th>\n",
       "<th>validation_pr_auc</th>\n",
       "<th>validation_lift</th>\n",
       "<th>validation_classification_error</th></tr></thead>\n",
       "    <tbody><tr><td></td>\n",
       "<td>2022-10-28 14:19:27</td>\n",
       "<td> 0.031 sec</td>\n",
       "<td>0.0</td>\n",
       "<td>0.5</td>\n",
       "<td>0.6931472</td>\n",
       "<td>0.5</td>\n",
       "<td>0.5022800</td>\n",
       "<td>1.0</td>\n",
       "<td>0.49772</td>\n",
       "<td>0.5</td>\n",
       "<td>0.6931472</td>\n",
       "<td>0.5</td>\n",
       "<td>0.4926400</td>\n",
       "<td>1.0</td>\n",
       "<td>0.50736</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:27</td>\n",
       "<td> 0.150 sec</td>\n",
       "<td>1.0</td>\n",
       "<td>0.3996010</td>\n",
       "<td>0.5084556</td>\n",
       "<td>0.9599605</td>\n",
       "<td>0.9528824</td>\n",
       "<td>1.9699443</td>\n",
       "<td>0.08508</td>\n",
       "<td>0.4002939</td>\n",
       "<td>0.5095766</td>\n",
       "<td>0.9584754</td>\n",
       "<td>0.9501212</td>\n",
       "<td>2.0073256</td>\n",
       "<td>0.08816</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:27</td>\n",
       "<td> 0.213 sec</td>\n",
       "<td>2.0</td>\n",
       "<td>0.3370778</td>\n",
       "<td>0.4032048</td>\n",
       "<td>0.9693358</td>\n",
       "<td>0.9638432</td>\n",
       "<td>1.9749781</td>\n",
       "<td>0.0786933</td>\n",
       "<td>0.3378726</td>\n",
       "<td>0.4043148</td>\n",
       "<td>0.9689642</td>\n",
       "<td>0.9627452</td>\n",
       "<td>2.0180782</td>\n",
       "<td>0.08072</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:27</td>\n",
       "<td> 0.272 sec</td>\n",
       "<td>3.0</td>\n",
       "<td>0.2957135</td>\n",
       "<td>0.3335651</td>\n",
       "<td>0.9735764</td>\n",
       "<td>0.9692971</td>\n",
       "<td>1.9823583</td>\n",
       "<td>0.07124</td>\n",
       "<td>0.2972804</td>\n",
       "<td>0.3355690</td>\n",
       "<td>0.9727585</td>\n",
       "<td>0.9672690</td>\n",
       "<td>2.0164813</td>\n",
       "<td>0.07328</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:27</td>\n",
       "<td> 0.337 sec</td>\n",
       "<td>4.0</td>\n",
       "<td>0.2690565</td>\n",
       "<td>0.2867050</td>\n",
       "<td>0.9769386</td>\n",
       "<td>0.9736661</td>\n",
       "<td>1.9824404</td>\n",
       "<td>0.0687733</td>\n",
       "<td>0.2711930</td>\n",
       "<td>0.2894303</td>\n",
       "<td>0.9753680</td>\n",
       "<td>0.9709177</td>\n",
       "<td>2.0166126</td>\n",
       "<td>0.07028</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:28</td>\n",
       "<td> 0.461 sec</td>\n",
       "<td>5.0</td>\n",
       "<td>0.2529770</td>\n",
       "<td>0.2548994</td>\n",
       "<td>0.9782349</td>\n",
       "<td>0.9753981</td>\n",
       "<td>1.9830013</td>\n",
       "<td>0.06936</td>\n",
       "<td>0.2550437</td>\n",
       "<td>0.2575265</td>\n",
       "<td>0.9769549</td>\n",
       "<td>0.9724197</td>\n",
       "<td>2.0204531</td>\n",
       "<td>0.06956</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:28</td>\n",
       "<td> 0.553 sec</td>\n",
       "<td>6.0</td>\n",
       "<td>0.2430550</td>\n",
       "<td>0.2328859</td>\n",
       "<td>0.9792536</td>\n",
       "<td>0.9767333</td>\n",
       "<td>1.9826945</td>\n",
       "<td>0.06864</td>\n",
       "<td>0.2452571</td>\n",
       "<td>0.2357295</td>\n",
       "<td>0.9778085</td>\n",
       "<td>0.9737537</td>\n",
       "<td>2.0198804</td>\n",
       "<td>0.06872</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:28</td>\n",
       "<td> 0.646 sec</td>\n",
       "<td>7.0</td>\n",
       "<td>0.2366703</td>\n",
       "<td>0.2173043</td>\n",
       "<td>0.9801005</td>\n",
       "<td>0.9779518</td>\n",
       "<td>1.9840125</td>\n",
       "<td>0.0655333</td>\n",
       "<td>0.2388924</td>\n",
       "<td>0.2202064</td>\n",
       "<td>0.9787945</td>\n",
       "<td>0.9751087</td>\n",
       "<td>2.0189075</td>\n",
       "<td>0.06696</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:28</td>\n",
       "<td> 0.742 sec</td>\n",
       "<td>8.0</td>\n",
       "<td>0.2274282</td>\n",
       "<td>0.2004642</td>\n",
       "<td>0.9816894</td>\n",
       "<td>0.9795849</td>\n",
       "<td>1.9883425</td>\n",
       "<td>0.06172</td>\n",
       "<td>0.2302864</td>\n",
       "<td>0.2040544</td>\n",
       "<td>0.9803330</td>\n",
       "<td>0.9772714</td>\n",
       "<td>2.0219506</td>\n",
       "<td>0.06516</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:28</td>\n",
       "<td> 0.831 sec</td>\n",
       "<td>9.0</td>\n",
       "<td>0.2235048</td>\n",
       "<td>0.1908878</td>\n",
       "<td>0.9823444</td>\n",
       "<td>0.9805219</td>\n",
       "<td>1.9883948</td>\n",
       "<td>0.0608</td>\n",
       "<td>0.2267565</td>\n",
       "<td>0.1951048</td>\n",
       "<td>0.9808128</td>\n",
       "<td>0.9779093</td>\n",
       "<td>2.0222773</td>\n",
       "<td>0.06432</td></tr>\n",
       "<tr><td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:28</td>\n",
       "<td> 1.227 sec</td>\n",
       "<td>13.0</td>\n",
       "<td>0.2093299</td>\n",
       "<td>0.1618817</td>\n",
       "<td>0.9849208</td>\n",
       "<td>0.9835753</td>\n",
       "<td>1.9885342</td>\n",
       "<td>0.0549333</td>\n",
       "<td>0.2140217</td>\n",
       "<td>0.1682975</td>\n",
       "<td>0.9829089</td>\n",
       "<td>0.9803703</td>\n",
       "<td>2.0227574</td>\n",
       "<td>0.0584</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:28</td>\n",
       "<td> 1.321 sec</td>\n",
       "<td>14.0</td>\n",
       "<td>0.2080116</td>\n",
       "<td>0.1587226</td>\n",
       "<td>0.9852452</td>\n",
       "<td>0.9839538</td>\n",
       "<td>1.9883086</td>\n",
       "<td>0.0544</td>\n",
       "<td>0.2129618</td>\n",
       "<td>0.1656925</td>\n",
       "<td>0.9831326</td>\n",
       "<td>0.9806934</td>\n",
       "<td>2.0222773</td>\n",
       "<td>0.05852</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:29</td>\n",
       "<td> 1.417 sec</td>\n",
       "<td>15.0</td>\n",
       "<td>0.2065626</td>\n",
       "<td>0.1556114</td>\n",
       "<td>0.9856595</td>\n",
       "<td>0.9846933</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0538533</td>\n",
       "<td>0.2118815</td>\n",
       "<td>0.1633915</td>\n",
       "<td>0.9833377</td>\n",
       "<td>0.9810162</td>\n",
       "<td>2.0227823</td>\n",
       "<td>0.05792</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:29</td>\n",
       "<td> 1.542 sec</td>\n",
       "<td>16.0</td>\n",
       "<td>0.2036096</td>\n",
       "<td>0.1517939</td>\n",
       "<td>0.9863474</td>\n",
       "<td>0.9853808</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0520267</td>\n",
       "<td>0.2099121</td>\n",
       "<td>0.1607776</td>\n",
       "<td>0.9837030</td>\n",
       "<td>0.9814828</td>\n",
       "<td>2.0217603</td>\n",
       "<td>0.0566</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:29</td>\n",
       "<td> 1.676 sec</td>\n",
       "<td>17.0</td>\n",
       "<td>0.2001280</td>\n",
       "<td>0.1470822</td>\n",
       "<td>0.9868868</td>\n",
       "<td>0.9859104</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0503733</td>\n",
       "<td>0.2068741</td>\n",
       "<td>0.1565508</td>\n",
       "<td>0.9842040</td>\n",
       "<td>0.9821555</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.05524</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:29</td>\n",
       "<td> 1.797 sec</td>\n",
       "<td>18.0</td>\n",
       "<td>0.1952364</td>\n",
       "<td>0.1409688</td>\n",
       "<td>0.9877667</td>\n",
       "<td>0.9868323</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0479067</td>\n",
       "<td>0.2029770</td>\n",
       "<td>0.1516665</td>\n",
       "<td>0.9848696</td>\n",
       "<td>0.9829746</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.05272</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:29</td>\n",
       "<td> 1.925 sec</td>\n",
       "<td>19.0</td>\n",
       "<td>0.1943690</td>\n",
       "<td>0.1391751</td>\n",
       "<td>0.9880068</td>\n",
       "<td>0.9871283</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0476667</td>\n",
       "<td>0.2025051</td>\n",
       "<td>0.1505835</td>\n",
       "<td>0.9849703</td>\n",
       "<td>0.9829211</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.05232</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:29</td>\n",
       "<td> 2.069 sec</td>\n",
       "<td>20.0</td>\n",
       "<td>0.1919489</td>\n",
       "<td>0.1361213</td>\n",
       "<td>0.9884007</td>\n",
       "<td>0.9875789</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0466933</td>\n",
       "<td>0.2009170</td>\n",
       "<td>0.1484152</td>\n",
       "<td>0.9852986</td>\n",
       "<td>0.9834889</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.05108</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:29</td>\n",
       "<td> 2.195 sec</td>\n",
       "<td>21.0</td>\n",
       "<td>0.1886257</td>\n",
       "<td>0.1319356</td>\n",
       "<td>0.9889611</td>\n",
       "<td>0.9882193</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0444533</td>\n",
       "<td>0.1983105</td>\n",
       "<td>0.1448947</td>\n",
       "<td>0.9857546</td>\n",
       "<td>0.9838308</td>\n",
       "<td>2.0217603</td>\n",
       "<td>0.0496</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:29</td>\n",
       "<td> 2.322 sec</td>\n",
       "<td>22.0</td>\n",
       "<td>0.1864343</td>\n",
       "<td>0.1290141</td>\n",
       "<td>0.9893340</td>\n",
       "<td>0.9887041</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0438133</td>\n",
       "<td>0.1964693</td>\n",
       "<td>0.1426421</td>\n",
       "<td>0.9860368</td>\n",
       "<td>0.9841777</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.049</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "<pre style='font-size: smaller; margin-bottom: 1em;'>[23 rows x 16 columns]</pre></div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-10.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-10 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-10 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-10 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-10 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-10 .h2o-table th,\n",
       "#h2o-table-10 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-10 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-10\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Variable Importances: </caption>\n",
       "    <thead><tr><th>variable</th>\n",
       "<th>relative_importance</th>\n",
       "<th>scaled_importance</th>\n",
       "<th>percentage</th></tr></thead>\n",
       "    <tbody><tr><td>C11</td>\n",
       "<td>70178.75</td>\n",
       "<td>1.0</td>\n",
       "<td>0.4261358</td></tr>\n",
       "<tr><td>C2</td>\n",
       "<td>35732.6640625</td>\n",
       "<td>0.5091664</td>\n",
       "<td>0.2169741</td></tr>\n",
       "<tr><td>C13</td>\n",
       "<td>21608.4863281</td>\n",
       "<td>0.3079064</td>\n",
       "<td>0.1312100</td></tr>\n",
       "<tr><td>C8</td>\n",
       "<td>15654.5498047</td>\n",
       "<td>0.2230668</td>\n",
       "<td>0.0950568</td></tr>\n",
       "<tr><td>C3</td>\n",
       "<td>12402.2607422</td>\n",
       "<td>0.1767239</td>\n",
       "<td>0.0753084</td></tr>\n",
       "<tr><td>C19</td>\n",
       "<td>5739.7641602</td>\n",
       "<td>0.0817878</td>\n",
       "<td>0.0348527</td></tr>\n",
       "<tr><td>C5</td>\n",
       "<td>2835.9191895</td>\n",
       "<td>0.0404099</td>\n",
       "<td>0.0172201</td></tr>\n",
       "<tr><td>C6</td>\n",
       "<td>88.9352493</td>\n",
       "<td>0.0012673</td>\n",
       "<td>0.0005400</td></tr>\n",
       "<tr><td>C12</td>\n",
       "<td>51.9876480</td>\n",
       "<td>0.0007408</td>\n",
       "<td>0.0003157</td></tr>\n",
       "<tr><td>C17</td>\n",
       "<td>49.1711044</td>\n",
       "<td>0.0007007</td>\n",
       "<td>0.0002986</td></tr>\n",
       "<tr><td>C7</td>\n",
       "<td>48.0521317</td>\n",
       "<td>0.0006847</td>\n",
       "<td>0.0002918</td></tr>\n",
       "<tr><td>C14</td>\n",
       "<td>46.5472183</td>\n",
       "<td>0.0006633</td>\n",
       "<td>0.0002826</td></tr>\n",
       "<tr><td>C10</td>\n",
       "<td>43.8788605</td>\n",
       "<td>0.0006252</td>\n",
       "<td>0.0002664</td></tr>\n",
       "<tr><td>C4</td>\n",
       "<td>42.4346542</td>\n",
       "<td>0.0006047</td>\n",
       "<td>0.0002577</td></tr>\n",
       "<tr><td>C18</td>\n",
       "<td>39.3952980</td>\n",
       "<td>0.0005614</td>\n",
       "<td>0.0002392</td></tr>\n",
       "<tr><td>C1</td>\n",
       "<td>32.5539932</td>\n",
       "<td>0.0004639</td>\n",
       "<td>0.0001977</td></tr>\n",
       "<tr><td>C15</td>\n",
       "<td>25.9528160</td>\n",
       "<td>0.0003698</td>\n",
       "<td>0.0001576</td></tr>\n",
       "<tr><td>C20</td>\n",
       "<td>23.1192322</td>\n",
       "<td>0.0003294</td>\n",
       "<td>0.0001404</td></tr>\n",
       "<tr><td>C16</td>\n",
       "<td>22.9983807</td>\n",
       "<td>0.0003277</td>\n",
       "<td>0.0001396</td></tr>\n",
       "<tr><td>C9</td>\n",
       "<td>18.9113159</td>\n",
       "<td>0.0002695</td>\n",
       "<td>0.0001148</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div><pre style=\"font-size: smaller; margin: 1em 0 0 0;\">\n",
       "\n",
       "[tips]\n",
       "Use `model.explain()` to inspect the model.\n",
       "--\n",
       "Use `h2o.display.toggle_user_tips()` to switch on/off this section.</pre>"
      ],
      "text/plain": [
       "Model Details\n",
       "=============\n",
       "H2OXGBoostEstimator : XGBoost\n",
       "Model Key: XGBoost_model_python_1666962310297_518\n",
       "\n",
       "\n",
       "Model Summary: \n",
       "    number_of_trees\n",
       "--  -----------------\n",
       "    22\n",
       "\n",
       "ModelMetricsBinomial: xgboost\n",
       "** Reported on train data. **\n",
       "\n",
       "MSE: 0.03475774769265435\n",
       "RMSE: 0.18643429859511995\n",
       "LogLoss: 0.12901406890397824\n",
       "Mean Per-Class Error: 0.04387881533186782\n",
       "AUC: 0.9893340459497513\n",
       "AUCPR: 0.988704085388505\n",
       "Gini: 0.9786680918995025\n",
       "\n",
       "Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.5101676438305829\n",
       "       0      1      Error    Rate\n",
       "-----  -----  -----  -------  ----------------\n",
       "0      35155  2174   0.0582   (2174.0/37329.0)\n",
       "1      1112   36559  0.0295   (1112.0/37671.0)\n",
       "Total  36267  38733  0.0438   (3286.0/75000.0)\n",
       "\n",
       "Maximum Metrics: Maximum metrics at their respective thresholds\n",
       "metric                       threshold    value     idx\n",
       "---------------------------  -----------  --------  -----\n",
       "max f1                       0.510168     0.956992  214\n",
       "max f2                       0.308281     0.971508  263\n",
       "max f0point5                 0.753261     0.959208  147\n",
       "max accuracy                 0.556171     0.9564    201\n",
       "max precision                0.997456     1         0\n",
       "max recall                   0.00160282   1         399\n",
       "max specificity              0.997456     1         0\n",
       "max absolute_mcc             0.556171     0.912928  201\n",
       "max min_per_class_accuracy   0.623985     0.954807  184\n",
       "max mean_per_class_accuracy  0.556171     0.95636   201\n",
       "max tns                      0.997456     37329     0\n",
       "max fns                      0.997456     37460     0\n",
       "max fps                      0.00160282   37329     399\n",
       "max tps                      0.00160282   37671     399\n",
       "max tnr                      0.997456     1         0\n",
       "max fnr                      0.997456     0.994399  0\n",
       "max fpr                      0.00160282   1         399\n",
       "max tpr                      0.00160282   1         399\n",
       "\n",
       "Gains/Lift Table: Avg response rate: 50.23 %, avg score: 50.22 %\n",
       "group    cumulative_data_fraction    lower_threshold    lift        cumulative_lift    response_rate    score       cumulative_response_rate    cumulative_score    capture_rate    cumulative_capture_rate    gain      cumulative_gain    kolmogorov_smirnov\n",
       "-------  --------------------------  -----------------  ----------  -----------------  ---------------  ----------  --------------------------  ------------------  --------------  -------------------------  --------  -----------------  --------------------\n",
       "1        0.0101733                   0.99611            1.99092     1.99092            1                0.996752    1                           0.996752            0.0202543       0.0202543                  99.0921   99.0921            0.0202543\n",
       "2        0.0200267                   0.995715           1.99092     1.99092            1                0.995889    1                           0.996327            0.0196172       0.0398715                  99.0921   99.0921            0.0398715\n",
       "3        0.03068                     0.995396           1.99092     1.99092            1                0.995549    1                           0.996057            0.0212099       0.0610815                  99.0921   99.0921            0.0610815\n",
       "4        0.0401067                   0.995194           1.98811     1.99026            0.998586         0.995271    0.999668                    0.995872            0.0187412       0.0798227                  98.8105   99.026             0.0797959\n",
       "5        0.0517733                   0.99493            1.98637     1.98938            0.997714         0.995008    0.999227                    0.995678            0.0231743       0.102997                   98.6371   98.9383            0.102917\n",
       "6        0.100013                    0.99331            1.98817     1.9888             0.998618         0.994166    0.998933                    0.994948            0.0959093       0.198906                   98.817    98.8798            0.198692\n",
       "7        0.15008                     0.990498           1.98403     1.98721            0.996538         0.992058    0.998134                    0.993984            0.0993337       0.29824                    98.4029   98.7207            0.297677\n",
       "8        0.200013                    0.98496            1.98242     1.98601            0.995728         0.987913    0.997533                    0.992468            0.0989886       0.397229                   98.2415   98.6011            0.396237\n",
       "9        0.30004                     0.956105           1.9673      1.97977            0.988136         0.974039    0.994401                    0.986324            0.196783        0.594011                   96.7302   97.9774            0.590636\n",
       "10       0.400013                    0.877061           1.91233     1.96292            0.960523         0.921008    0.985934                    0.97                0.191182        0.785193                   91.2325   96.2917            0.773888\n",
       "11       0.500013                    0.638348           1.67636     1.90561            0.842            0.805119    0.957148                    0.937025            0.167636        0.952828                   67.6356   90.5606            0.909779\n",
       "12       0.6                         0.0781245          0.426114    1.65906            0.214029         0.287618    0.833311                    0.828805            0.0426057       0.995434                   -57.3886  65.9057            0.794491\n",
       "13       0.700013                    0.0128222          0.0238879   1.42543            0.0119984        0.0339001   0.715967                    0.715234            0.00238911      0.997823                   -97.6112  42.5435            0.598348\n",
       "14       0.80008                     0.00579305         0.0122028   1.24868            0.00612925       0.00841651  0.627187                    0.626832            0.0012211       0.999044                   -98.7797  24.8681            0.399752\n",
       "15       0.9                         0.00344276         0.00451637  1.11055            0.00226848       0.00452987  0.557807                    0.557742            0.000451276     0.999496                   -99.5484  11.0551            0.199903\n",
       "16       1                           0.000167174        0.00504367  1                  0.00253333       0.00262682  0.50228                     0.502231            0.000504367     1                          -99.4956  0                  0\n",
       "\n",
       "ModelMetricsBinomial: xgboost\n",
       "** Reported on validation data. **\n",
       "\n",
       "MSE: 0.038600168581279404\n",
       "RMSE: 0.1964692560714765\n",
       "LogLoss: 0.14264211397729937\n",
       "Mean Per-Class Error: 0.04874265748103474\n",
       "AUC: 0.9860367984758349\n",
       "AUCPR: 0.9841777019906762\n",
       "Gini: 0.9720735969516698\n",
       "\n",
       "Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.4899955771186135\n",
       "       0      1      Error    Rate\n",
       "-----  -----  -----  -------  ----------------\n",
       "0      11844  840    0.0662   (840.0/12684.0)\n",
       "1      385    11931  0.0313   (385.0/12316.0)\n",
       "Total  12229  12771  0.049    (1225.0/25000.0)\n",
       "\n",
       "Maximum Metrics: Maximum metrics at their respective thresholds\n",
       "metric                       threshold    value     idx\n",
       "---------------------------  -----------  --------  -----\n",
       "max f1                       0.489996     0.95117   217\n",
       "max f2                       0.238164     0.968401  281\n",
       "max f0point5                 0.76376      0.952103  144\n",
       "max accuracy                 0.489996     0.951     217\n",
       "max precision                0.997468     1         0\n",
       "max recall                   0.00183535   1         398\n",
       "max specificity              0.997468     1         0\n",
       "max absolute_mcc             0.489996     0.902629  217\n",
       "max min_per_class_accuracy   0.630364     0.949497  182\n",
       "max mean_per_class_accuracy  0.489996     0.951257  217\n",
       "max tns                      0.997468     12684     0\n",
       "max fns                      0.997468     12244     0\n",
       "max fps                      0.00134346   12684     399\n",
       "max tps                      0.00183535   12316     398\n",
       "max tnr                      0.997468     1         0\n",
       "max fnr                      0.997468     0.994154  0\n",
       "max fpr                      0.00134346   1         399\n",
       "max tpr                      0.00183535   1         398\n",
       "\n",
       "Gains/Lift Table: Avg response rate: 49.26 %, avg score: 49.47 %\n",
       "group    cumulative_data_fraction    lower_threshold    lift        cumulative_lift    response_rate    score       cumulative_response_rate    cumulative_score    capture_rate    cumulative_capture_rate    gain      cumulative_gain    kolmogorov_smirnov\n",
       "-------  --------------------------  -----------------  ----------  -----------------  ---------------  ----------  --------------------------  ------------------  --------------  -------------------------  --------  -----------------  --------------------\n",
       "1        0.01028                     0.996124           2.02988     2.02988            1                0.996731    1                           0.996731            0.0208672       0.0208672                  102.988   102.988            0.0208672\n",
       "2        0.02032                     0.995801           2.02988     2.02988            1                0.995918    1                           0.996329            0.02038         0.0412472                  102.988   102.988            0.0412472\n",
       "3        0.03016                     0.995462           2.02163     2.02719            0.995935         0.99561     0.998674                    0.996095            0.0198928       0.06114                    102.163   102.719            0.0610611\n",
       "4        0.04108                     0.995194           2.02244     2.02593            0.996337         0.995289    0.998053                    0.99588             0.0220851       0.0832251                  102.244   102.593            0.0830674\n",
       "5        0.05256                     0.99493            2.01573     2.0237             0.993031         0.995009    0.996956                    0.99569             0.0231406       0.106366                   101.573   102.37             0.10605\n",
       "6        0.1                         0.993286           2.01961     2.02176            0.994941         0.994155    0.996                       0.994962            0.0958103       0.202176                   101.961   102.176            0.201388\n",
       "7        0.15                        0.990269           2.02338     2.0223             0.9968           0.991949    0.996267                    0.993958            0.101169        0.303345                   102.338   102.23             0.302241\n",
       "8        0.2                         0.984066           2.01364     2.02014            0.992            0.987394    0.9952                      0.992317            0.100682        0.404027                   101.364   102.014            0.402135\n",
       "9        0.3                         0.95239            1.98522     2.0085             0.978            0.971705    0.989467                    0.985446            0.198522        0.60255                    98.5222   100.85             0.596321\n",
       "10       0.4                         0.867106           1.92351     1.98725            0.9476           0.914257    0.979                       0.967649            0.192351        0.794901                   92.3514   98.7252            0.778345\n",
       "11       0.5                         0.582235           1.61741     1.91328            0.7968           0.782506    0.94256                     0.93062             0.161741        0.956642                   61.7408   91.3284            0.900035\n",
       "12       0.6                         0.0667765          0.377558    1.65733            0.186            0.248018    0.816467                    0.816853            0.0377558       0.994398                   -62.2442  65.7329            0.777352\n",
       "13       0.7                         0.0125771          0.0186749   1.42324            0.0092           0.0304482   0.701143                    0.70451             0.00186749      0.996265                   -98.1325  42.3236            0.583935\n",
       "14       0.80048                     0.00574124         0.0185857   1.24692            0.00915605       0.00815736  0.614281                    0.6171              0.00186749      0.998133                   -98.1414  24.6917            0.389571\n",
       "15       0.9014                      0.00345923         0.0104592   1.10848            0.0051526        0.0045211   0.546084                    0.548516            0.00105554      0.999188                   -98.9541  10.8485            0.192739\n",
       "16       1                           0.000260908        0.00823481  1                  0.0040568        0.00264459  0.49264                     0.494693            0.000811952     1                          -99.1765  0                  0\n",
       "\n",
       "Scoring History: \n",
       "     timestamp            duration    number_of_trees    training_rmse        training_logloss     training_auc        training_pr_auc     training_lift       training_classification_error    validation_rmse      validation_logloss    validation_auc      validation_pr_auc    validation_lift     validation_classification_error\n",
       "---  -------------------  ----------  -----------------  -------------------  -------------------  ------------------  ------------------  ------------------  -------------------------------  -------------------  --------------------  ------------------  -------------------  ------------------  ---------------------------------\n",
       "     2022-10-28 14:19:27  0.031 sec   0.0                0.5                  0.6931471805600504   0.5                 0.5022800000000001  1.0                 0.49772                          0.5                  0.6931471805599932    0.5                 0.49263999999999997  1.0                 0.50736\n",
       "     2022-10-28 14:19:27  0.150 sec   1.0                0.39960097366259284  0.5084555759713797   0.9599604712562773  0.952882361784991   1.969944257252065   0.08508                          0.4002938936840949   0.5095766191321826    0.9584754441256724  0.9501212341159861   2.007325610768287   0.08816\n",
       "     2022-10-28 14:19:27  0.213 sec   2.0                0.3370778235169717   0.40320483936633433  0.9693357563355386  0.9638432211755918  1.974978083921303   0.07869333333333334              0.3378725612813438   0.4043147576521532    0.9689642384208382  0.9627452021209266   2.0180782041889165  0.08072\n",
       "     2022-10-28 14:19:27  0.272 sec   3.0                0.2957135469964664   0.33356511573213676  0.9735764390034864  0.9692971186564519  1.9823582956342736  0.07124                          0.2972804020126436   0.33556896107331324   0.972758545365196   0.9672690232547145   2.0164812843739717  0.07328\n",
       "     2022-10-28 14:19:27  0.337 sec   4.0                0.2690565369849074   0.2867049786815229   0.9769385896258127  0.973666085141308   1.9824403701018989  0.06877333333333334              0.2711929535273588   0.2894303318159203    0.9753679683707979  0.9709177118281687   2.0166126426753443  0.07028\n",
       "     2022-10-28 14:19:28  0.461 sec   5.0                0.2529770080480673   0.25489941558805895  0.9782348903583494  0.9753981493946517  1.9830012735164846  0.06936                          0.2550436637594876   0.25752647933993555   0.976954942633842   0.9724197019498086   2.0204531445917966  0.06956\n",
       "     2022-10-28 14:19:28  0.553 sec   6.0                0.24305497030792145  0.23288588902305773  0.9792535639846858  0.9767332522440302  1.982694450495822   0.06864                          0.24525713259217433  0.23572949157271123   0.9778084651737403  0.9737536794534183   2.0198804230789538  0.06872\n",
       "     2022-10-28 14:19:28  0.646 sec   7.0                0.23667027877522748  0.21730431422414767  0.9801004932398384  0.9779517651384936  1.9840124751203916  0.06553333333333333              0.2388923863271564   0.22020638093402833   0.978794451615705   0.9751086853604877   2.018907507702571   0.06696\n",
       "     2022-10-28 14:19:28  0.742 sec   8.0                0.22742820294228833  0.2004641682608729   0.981689446457674   0.979584885842671   1.988342484694663   0.06172                          0.23028641120487947  0.20405435411113004   0.9803330089878547  0.9772714003156485   2.021950613023709   0.06516\n",
       "     2022-10-28 14:19:28  0.831 sec   9.0                0.22350478838486162  0.1908877726426607   0.9823443926985863  0.9805219447464963  1.9883948484251914  0.0608                           0.22675651465031937  0.19510476775550073   0.9808127993480623  0.9779093493480279   2.0222772849300505  0.06432\n",
       "---  ---                  ---         ---                ---                  ---                  ---                 ---                 ---                 ---                              ---                  ---                   ---                 ---                  ---                 ---\n",
       "     2022-10-28 14:19:28  1.227 sec   13.0               0.20932986426153485  0.16188173031435207  0.9849208281386209  0.9835752931336927  1.9885342025018198  0.054933333333333334             0.2140216611245386   0.16829754841446237   0.9829088919260484  0.9803702989731918   2.022757445741668   0.0584\n",
       "     2022-10-28 14:19:28  1.321 sec   14.0               0.20801159542673212  0.15872260659114573  0.9852451840387032  0.9839537573047635  1.9883086406824775  0.0544                           0.2129618210780405   0.16569253851178234   0.9831326172024832  0.9806933766171955   2.0222772849300505  0.05852\n",
       "     2022-10-28 14:19:29  1.417 sec   15.0               0.20656261820821578  0.15561137441472306  0.9856594600307703  0.9846933313933812  1.9909213984231904  0.053853333333333336             0.2118814635733488   0.16339145578701472   0.9833376856363834  0.9810162497973599   2.022782349187026   0.05792\n",
       "     2022-10-28 14:19:29  1.542 sec   16.0               0.2036095613587592   0.1517938621462097   0.9863473648236762  0.9853807508666774  1.9909213984231904  0.052026666666666666             0.20991212994517833  0.1607776039482004    0.9837030416011293  0.9814827764043785   2.0217603117895417  0.0566\n",
       "     2022-10-28 14:19:29  1.676 sec   17.0               0.20012801070573083  0.1470822013139249   0.986886830974453   0.985910377892329   1.9909213984231904  0.05037333333333333              0.2068740603495647   0.1565507907729844    0.9842040109503664  0.9821554851954529   2.029879831113998   0.05524\n",
       "     2022-10-28 14:19:29  1.797 sec   18.0               0.1952363704640891   0.14096879922102418  0.9877666946033187  0.9868323340675447  1.9909213984231904  0.04790666666666667              0.20297695900076984  0.15166647071587894   0.9848695695625415  0.9829746186958338   2.029879831113998   0.05272\n",
       "     2022-10-28 14:19:29  1.925 sec   19.0               0.19436898343164097  0.1391751330056711   0.9880068361300588  0.9871283090191609  1.9909213984231904  0.04766666666666667              0.2025051413752741   0.15058346705393647   0.9849703081904262  0.9829211071563434   2.029879831113998   0.05232\n",
       "     2022-10-28 14:19:29  2.069 sec   20.0               0.19194891175623852  0.13612128709347052  0.988400723075942   0.9875788628040455  1.9909213984231904  0.04669333333333334              0.2009170028210971   0.14841515569437927   0.9852986065255842  0.9834889458246532   2.029879831113998   0.05108\n",
       "     2022-10-28 14:19:29  2.195 sec   21.0               0.18862571988282295  0.1319356174549523   0.9889610721498387  0.9882193402029281  1.9909213984231904  0.04445333333333333              0.1983104566213673   0.1448947109368754    0.9857545901273815  0.9838307814948767   2.0217603117895417  0.0496\n",
       "     2022-10-28 14:19:29  2.322 sec   22.0               0.18643429859511995  0.12901406890397824  0.9893340459497513  0.988704085388505   1.9909213984231904  0.043813333333333336             0.1964692560714765   0.14264211397729937   0.9860367984758349  0.9841777019906762   2.029879831113998   0.049\n",
       "[23 rows x 16 columns]\n",
       "\n",
       "\n",
       "Variable Importances: \n",
       "variable    relative_importance    scaled_importance    percentage\n",
       "----------  ---------------------  -------------------  ------------\n",
       "C11         70178.8                1                    0.426136\n",
       "C2          35732.7                0.509166             0.216974\n",
       "C13         21608.5                0.307906             0.13121\n",
       "C8          15654.5                0.223067             0.0950568\n",
       "C3          12402.3                0.176724             0.0753084\n",
       "C19         5739.76                0.0817878            0.0348527\n",
       "C5          2835.92                0.0404099            0.0172201\n",
       "C6          88.9352                0.00126727           0.000540028\n",
       "C12         51.9876                0.000740789          0.000315677\n",
       "C17         49.1711                0.000700655          0.000298574\n",
       "C7          48.0521                0.000684711          0.00029178\n",
       "C14         46.5472                0.000663267          0.000282642\n",
       "C10         43.8789                0.000625244          0.000266439\n",
       "C4          42.4347                0.000604665          0.00025767\n",
       "C18         39.3953                0.000561357          0.000239214\n",
       "C1          32.554                 0.000463873          0.000197673\n",
       "C15         25.9528                0.00036981           0.000157589\n",
       "C20         23.1192                0.000329434          0.000140383\n",
       "C16         22.9984                0.000327711          0.00013965\n",
       "C9          18.9113                0.000269474          0.000114832\n",
       "\n",
       "[tips]\n",
       "Use `model.explain()` to inspect the model.\n",
       "--\n",
       "Use `h2o.display.toggle_user_tips()` to switch on/off this section."
      ]
     },
     "execution_count": 5,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from h2o.estimators.xgboost import H2OXGBoostEstimator\n",
    "model_def = H2OXGBoostEstimator(ntrees=1000, max_depth=6, score_each_iteration=True, \n",
    "                                stopping_rounds=3, stopping_tolerance=0.05, stopping_metric=\"logloss\")\n",
    "model_def.train(y=\"y\", training_frame=train, validation_frame=valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7e5c5e83",
   "metadata": {},
   "source": [
    "Scoring history will show us some of the metrics that were calculated on training and validation datasets. In our case, `validation_logloss` was used as the metric for early stopping."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "798f28fd",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>duration</th>\n",
       "      <th>number_of_trees</th>\n",
       "      <th>training_rmse</th>\n",
       "      <th>training_logloss</th>\n",
       "      <th>training_auc</th>\n",
       "      <th>training_pr_auc</th>\n",
       "      <th>training_lift</th>\n",
       "      <th>training_classification_error</th>\n",
       "      <th>validation_rmse</th>\n",
       "      <th>validation_logloss</th>\n",
       "      <th>validation_auc</th>\n",
       "      <th>validation_pr_auc</th>\n",
       "      <th>validation_lift</th>\n",
       "      <th>validation_classification_error</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:27</td>\n",
       "      <td>0.031 sec</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.693147</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.502280</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.497720</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.693147</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.492640</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.50736</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:27</td>\n",
       "      <td>0.150 sec</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.399601</td>\n",
       "      <td>0.508456</td>\n",
       "      <td>0.959960</td>\n",
       "      <td>0.952882</td>\n",
       "      <td>1.969944</td>\n",
       "      <td>0.085080</td>\n",
       "      <td>0.400294</td>\n",
       "      <td>0.509577</td>\n",
       "      <td>0.958475</td>\n",
       "      <td>0.950121</td>\n",
       "      <td>2.007326</td>\n",
       "      <td>0.08816</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:27</td>\n",
       "      <td>0.213 sec</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.337078</td>\n",
       "      <td>0.403205</td>\n",
       "      <td>0.969336</td>\n",
       "      <td>0.963843</td>\n",
       "      <td>1.974978</td>\n",
       "      <td>0.078693</td>\n",
       "      <td>0.337873</td>\n",
       "      <td>0.404315</td>\n",
       "      <td>0.968964</td>\n",
       "      <td>0.962745</td>\n",
       "      <td>2.018078</td>\n",
       "      <td>0.08072</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:27</td>\n",
       "      <td>0.272 sec</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.295714</td>\n",
       "      <td>0.333565</td>\n",
       "      <td>0.973576</td>\n",
       "      <td>0.969297</td>\n",
       "      <td>1.982358</td>\n",
       "      <td>0.071240</td>\n",
       "      <td>0.297280</td>\n",
       "      <td>0.335569</td>\n",
       "      <td>0.972759</td>\n",
       "      <td>0.967269</td>\n",
       "      <td>2.016481</td>\n",
       "      <td>0.07328</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:27</td>\n",
       "      <td>0.337 sec</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.269057</td>\n",
       "      <td>0.286705</td>\n",
       "      <td>0.976939</td>\n",
       "      <td>0.973666</td>\n",
       "      <td>1.982440</td>\n",
       "      <td>0.068773</td>\n",
       "      <td>0.271193</td>\n",
       "      <td>0.289430</td>\n",
       "      <td>0.975368</td>\n",
       "      <td>0.970918</td>\n",
       "      <td>2.016613</td>\n",
       "      <td>0.07028</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>0.461 sec</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.252977</td>\n",
       "      <td>0.254899</td>\n",
       "      <td>0.978235</td>\n",
       "      <td>0.975398</td>\n",
       "      <td>1.983001</td>\n",
       "      <td>0.069360</td>\n",
       "      <td>0.255044</td>\n",
       "      <td>0.257526</td>\n",
       "      <td>0.976955</td>\n",
       "      <td>0.972420</td>\n",
       "      <td>2.020453</td>\n",
       "      <td>0.06956</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>0.553 sec</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.243055</td>\n",
       "      <td>0.232886</td>\n",
       "      <td>0.979254</td>\n",
       "      <td>0.976733</td>\n",
       "      <td>1.982694</td>\n",
       "      <td>0.068640</td>\n",
       "      <td>0.245257</td>\n",
       "      <td>0.235729</td>\n",
       "      <td>0.977808</td>\n",
       "      <td>0.973754</td>\n",
       "      <td>2.019880</td>\n",
       "      <td>0.06872</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>0.646 sec</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.236670</td>\n",
       "      <td>0.217304</td>\n",
       "      <td>0.980100</td>\n",
       "      <td>0.977952</td>\n",
       "      <td>1.984012</td>\n",
       "      <td>0.065533</td>\n",
       "      <td>0.238892</td>\n",
       "      <td>0.220206</td>\n",
       "      <td>0.978794</td>\n",
       "      <td>0.975109</td>\n",
       "      <td>2.018908</td>\n",
       "      <td>0.06696</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>0.742 sec</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.227428</td>\n",
       "      <td>0.200464</td>\n",
       "      <td>0.981689</td>\n",
       "      <td>0.979585</td>\n",
       "      <td>1.988342</td>\n",
       "      <td>0.061720</td>\n",
       "      <td>0.230286</td>\n",
       "      <td>0.204054</td>\n",
       "      <td>0.980333</td>\n",
       "      <td>0.977271</td>\n",
       "      <td>2.021951</td>\n",
       "      <td>0.06516</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>0.831 sec</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0.223505</td>\n",
       "      <td>0.190888</td>\n",
       "      <td>0.982344</td>\n",
       "      <td>0.980522</td>\n",
       "      <td>1.988395</td>\n",
       "      <td>0.060800</td>\n",
       "      <td>0.226757</td>\n",
       "      <td>0.195105</td>\n",
       "      <td>0.980813</td>\n",
       "      <td>0.977909</td>\n",
       "      <td>2.022277</td>\n",
       "      <td>0.06432</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>0.940 sec</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.220040</td>\n",
       "      <td>0.182967</td>\n",
       "      <td>0.983000</td>\n",
       "      <td>0.981210</td>\n",
       "      <td>1.988298</td>\n",
       "      <td>0.059227</td>\n",
       "      <td>0.223656</td>\n",
       "      <td>0.187705</td>\n",
       "      <td>0.981356</td>\n",
       "      <td>0.978627</td>\n",
       "      <td>2.021888</td>\n",
       "      <td>0.06240</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>1.039 sec</td>\n",
       "      <td>11.0</td>\n",
       "      <td>0.214926</td>\n",
       "      <td>0.173463</td>\n",
       "      <td>0.983878</td>\n",
       "      <td>0.982224</td>\n",
       "      <td>1.988819</td>\n",
       "      <td>0.057027</td>\n",
       "      <td>0.218899</td>\n",
       "      <td>0.178748</td>\n",
       "      <td>0.982148</td>\n",
       "      <td>0.979434</td>\n",
       "      <td>2.023615</td>\n",
       "      <td>0.06084</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>1.135 sec</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.212074</td>\n",
       "      <td>0.167261</td>\n",
       "      <td>0.984348</td>\n",
       "      <td>0.982791</td>\n",
       "      <td>1.988617</td>\n",
       "      <td>0.055987</td>\n",
       "      <td>0.216237</td>\n",
       "      <td>0.172957</td>\n",
       "      <td>0.982491</td>\n",
       "      <td>0.979711</td>\n",
       "      <td>2.022975</td>\n",
       "      <td>0.05932</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>1.227 sec</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.209330</td>\n",
       "      <td>0.161882</td>\n",
       "      <td>0.984921</td>\n",
       "      <td>0.983575</td>\n",
       "      <td>1.988534</td>\n",
       "      <td>0.054933</td>\n",
       "      <td>0.214022</td>\n",
       "      <td>0.168298</td>\n",
       "      <td>0.982909</td>\n",
       "      <td>0.980370</td>\n",
       "      <td>2.022757</td>\n",
       "      <td>0.05840</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:28</td>\n",
       "      <td>1.321 sec</td>\n",
       "      <td>14.0</td>\n",
       "      <td>0.208012</td>\n",
       "      <td>0.158723</td>\n",
       "      <td>0.985245</td>\n",
       "      <td>0.983954</td>\n",
       "      <td>1.988309</td>\n",
       "      <td>0.054400</td>\n",
       "      <td>0.212962</td>\n",
       "      <td>0.165693</td>\n",
       "      <td>0.983133</td>\n",
       "      <td>0.980693</td>\n",
       "      <td>2.022277</td>\n",
       "      <td>0.05852</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:29</td>\n",
       "      <td>1.417 sec</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0.206563</td>\n",
       "      <td>0.155611</td>\n",
       "      <td>0.985659</td>\n",
       "      <td>0.984693</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.053853</td>\n",
       "      <td>0.211881</td>\n",
       "      <td>0.163391</td>\n",
       "      <td>0.983338</td>\n",
       "      <td>0.981016</td>\n",
       "      <td>2.022782</td>\n",
       "      <td>0.05792</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:29</td>\n",
       "      <td>1.542 sec</td>\n",
       "      <td>16.0</td>\n",
       "      <td>0.203610</td>\n",
       "      <td>0.151794</td>\n",
       "      <td>0.986347</td>\n",
       "      <td>0.985381</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.052027</td>\n",
       "      <td>0.209912</td>\n",
       "      <td>0.160778</td>\n",
       "      <td>0.983703</td>\n",
       "      <td>0.981483</td>\n",
       "      <td>2.021760</td>\n",
       "      <td>0.05660</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:29</td>\n",
       "      <td>1.676 sec</td>\n",
       "      <td>17.0</td>\n",
       "      <td>0.200128</td>\n",
       "      <td>0.147082</td>\n",
       "      <td>0.986887</td>\n",
       "      <td>0.985910</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.050373</td>\n",
       "      <td>0.206874</td>\n",
       "      <td>0.156551</td>\n",
       "      <td>0.984204</td>\n",
       "      <td>0.982155</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.05524</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:29</td>\n",
       "      <td>1.797 sec</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0.195236</td>\n",
       "      <td>0.140969</td>\n",
       "      <td>0.987767</td>\n",
       "      <td>0.986832</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.047907</td>\n",
       "      <td>0.202977</td>\n",
       "      <td>0.151666</td>\n",
       "      <td>0.984870</td>\n",
       "      <td>0.982975</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.05272</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:29</td>\n",
       "      <td>1.925 sec</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0.194369</td>\n",
       "      <td>0.139175</td>\n",
       "      <td>0.988007</td>\n",
       "      <td>0.987128</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.047667</td>\n",
       "      <td>0.202505</td>\n",
       "      <td>0.150583</td>\n",
       "      <td>0.984970</td>\n",
       "      <td>0.982921</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.05232</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:29</td>\n",
       "      <td>2.069 sec</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.191949</td>\n",
       "      <td>0.136121</td>\n",
       "      <td>0.988401</td>\n",
       "      <td>0.987579</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.046693</td>\n",
       "      <td>0.200917</td>\n",
       "      <td>0.148415</td>\n",
       "      <td>0.985299</td>\n",
       "      <td>0.983489</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.05108</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:29</td>\n",
       "      <td>2.195 sec</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0.188626</td>\n",
       "      <td>0.131936</td>\n",
       "      <td>0.988961</td>\n",
       "      <td>0.988219</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.044453</td>\n",
       "      <td>0.198310</td>\n",
       "      <td>0.144895</td>\n",
       "      <td>0.985755</td>\n",
       "      <td>0.983831</td>\n",
       "      <td>2.021760</td>\n",
       "      <td>0.04960</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:29</td>\n",
       "      <td>2.322 sec</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.186434</td>\n",
       "      <td>0.129014</td>\n",
       "      <td>0.989334</td>\n",
       "      <td>0.988704</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.043813</td>\n",
       "      <td>0.196469</td>\n",
       "      <td>0.142642</td>\n",
       "      <td>0.986037</td>\n",
       "      <td>0.984178</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.04900</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                timestamp    duration  number_of_trees  training_rmse  \\\n",
       "0     2022-10-28 14:19:27   0.031 sec              0.0       0.500000   \n",
       "1     2022-10-28 14:19:27   0.150 sec              1.0       0.399601   \n",
       "2     2022-10-28 14:19:27   0.213 sec              2.0       0.337078   \n",
       "3     2022-10-28 14:19:27   0.272 sec              3.0       0.295714   \n",
       "4     2022-10-28 14:19:27   0.337 sec              4.0       0.269057   \n",
       "5     2022-10-28 14:19:28   0.461 sec              5.0       0.252977   \n",
       "6     2022-10-28 14:19:28   0.553 sec              6.0       0.243055   \n",
       "7     2022-10-28 14:19:28   0.646 sec              7.0       0.236670   \n",
       "8     2022-10-28 14:19:28   0.742 sec              8.0       0.227428   \n",
       "9     2022-10-28 14:19:28   0.831 sec              9.0       0.223505   \n",
       "10    2022-10-28 14:19:28   0.940 sec             10.0       0.220040   \n",
       "11    2022-10-28 14:19:28   1.039 sec             11.0       0.214926   \n",
       "12    2022-10-28 14:19:28   1.135 sec             12.0       0.212074   \n",
       "13    2022-10-28 14:19:28   1.227 sec             13.0       0.209330   \n",
       "14    2022-10-28 14:19:28   1.321 sec             14.0       0.208012   \n",
       "15    2022-10-28 14:19:29   1.417 sec             15.0       0.206563   \n",
       "16    2022-10-28 14:19:29   1.542 sec             16.0       0.203610   \n",
       "17    2022-10-28 14:19:29   1.676 sec             17.0       0.200128   \n",
       "18    2022-10-28 14:19:29   1.797 sec             18.0       0.195236   \n",
       "19    2022-10-28 14:19:29   1.925 sec             19.0       0.194369   \n",
       "20    2022-10-28 14:19:29   2.069 sec             20.0       0.191949   \n",
       "21    2022-10-28 14:19:29   2.195 sec             21.0       0.188626   \n",
       "22    2022-10-28 14:19:29   2.322 sec             22.0       0.186434   \n",
       "\n",
       "    training_logloss  training_auc  training_pr_auc  training_lift  \\\n",
       "0           0.693147      0.500000         0.502280       1.000000   \n",
       "1           0.508456      0.959960         0.952882       1.969944   \n",
       "2           0.403205      0.969336         0.963843       1.974978   \n",
       "3           0.333565      0.973576         0.969297       1.982358   \n",
       "4           0.286705      0.976939         0.973666       1.982440   \n",
       "5           0.254899      0.978235         0.975398       1.983001   \n",
       "6           0.232886      0.979254         0.976733       1.982694   \n",
       "7           0.217304      0.980100         0.977952       1.984012   \n",
       "8           0.200464      0.981689         0.979585       1.988342   \n",
       "9           0.190888      0.982344         0.980522       1.988395   \n",
       "10          0.182967      0.983000         0.981210       1.988298   \n",
       "11          0.173463      0.983878         0.982224       1.988819   \n",
       "12          0.167261      0.984348         0.982791       1.988617   \n",
       "13          0.161882      0.984921         0.983575       1.988534   \n",
       "14          0.158723      0.985245         0.983954       1.988309   \n",
       "15          0.155611      0.985659         0.984693       1.990921   \n",
       "16          0.151794      0.986347         0.985381       1.990921   \n",
       "17          0.147082      0.986887         0.985910       1.990921   \n",
       "18          0.140969      0.987767         0.986832       1.990921   \n",
       "19          0.139175      0.988007         0.987128       1.990921   \n",
       "20          0.136121      0.988401         0.987579       1.990921   \n",
       "21          0.131936      0.988961         0.988219       1.990921   \n",
       "22          0.129014      0.989334         0.988704       1.990921   \n",
       "\n",
       "    training_classification_error  validation_rmse  validation_logloss  \\\n",
       "0                        0.497720         0.500000            0.693147   \n",
       "1                        0.085080         0.400294            0.509577   \n",
       "2                        0.078693         0.337873            0.404315   \n",
       "3                        0.071240         0.297280            0.335569   \n",
       "4                        0.068773         0.271193            0.289430   \n",
       "5                        0.069360         0.255044            0.257526   \n",
       "6                        0.068640         0.245257            0.235729   \n",
       "7                        0.065533         0.238892            0.220206   \n",
       "8                        0.061720         0.230286            0.204054   \n",
       "9                        0.060800         0.226757            0.195105   \n",
       "10                       0.059227         0.223656            0.187705   \n",
       "11                       0.057027         0.218899            0.178748   \n",
       "12                       0.055987         0.216237            0.172957   \n",
       "13                       0.054933         0.214022            0.168298   \n",
       "14                       0.054400         0.212962            0.165693   \n",
       "15                       0.053853         0.211881            0.163391   \n",
       "16                       0.052027         0.209912            0.160778   \n",
       "17                       0.050373         0.206874            0.156551   \n",
       "18                       0.047907         0.202977            0.151666   \n",
       "19                       0.047667         0.202505            0.150583   \n",
       "20                       0.046693         0.200917            0.148415   \n",
       "21                       0.044453         0.198310            0.144895   \n",
       "22                       0.043813         0.196469            0.142642   \n",
       "\n",
       "    validation_auc  validation_pr_auc  validation_lift  \\\n",
       "0         0.500000           0.492640         1.000000   \n",
       "1         0.958475           0.950121         2.007326   \n",
       "2         0.968964           0.962745         2.018078   \n",
       "3         0.972759           0.967269         2.016481   \n",
       "4         0.975368           0.970918         2.016613   \n",
       "5         0.976955           0.972420         2.020453   \n",
       "6         0.977808           0.973754         2.019880   \n",
       "7         0.978794           0.975109         2.018908   \n",
       "8         0.980333           0.977271         2.021951   \n",
       "9         0.980813           0.977909         2.022277   \n",
       "10        0.981356           0.978627         2.021888   \n",
       "11        0.982148           0.979434         2.023615   \n",
       "12        0.982491           0.979711         2.022975   \n",
       "13        0.982909           0.980370         2.022757   \n",
       "14        0.983133           0.980693         2.022277   \n",
       "15        0.983338           0.981016         2.022782   \n",
       "16        0.983703           0.981483         2.021760   \n",
       "17        0.984204           0.982155         2.029880   \n",
       "18        0.984870           0.982975         2.029880   \n",
       "19        0.984970           0.982921         2.029880   \n",
       "20        0.985299           0.983489         2.029880   \n",
       "21        0.985755           0.983831         2.021760   \n",
       "22        0.986037           0.984178         2.029880   \n",
       "\n",
       "    validation_classification_error  \n",
       "0                           0.50736  \n",
       "1                           0.08816  \n",
       "2                           0.08072  \n",
       "3                           0.07328  \n",
       "4                           0.07028  \n",
       "5                           0.06956  \n",
       "6                           0.06872  \n",
       "7                           0.06696  \n",
       "8                           0.06516  \n",
       "9                           0.06432  \n",
       "10                          0.06240  \n",
       "11                          0.06084  \n",
       "12                          0.05932  \n",
       "13                          0.05840  \n",
       "14                          0.05852  \n",
       "15                          0.05792  \n",
       "16                          0.05660  \n",
       "17                          0.05524  \n",
       "18                          0.05272  \n",
       "19                          0.05232  \n",
       "20                          0.05108  \n",
       "21                          0.04960  \n",
       "22                          0.04900  "
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_def.scoring_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "45b72bf5",
   "metadata": {},
   "source": [
    "##### Train XGBoost model and use eval_metric=\"logloss\" for early stopping"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "59bcee0a",
   "metadata": {},
   "source": [
    "We will use the same parameters as in the first case and add `eval_metric=\"logloss\"`. To use actually use the value of the `eval_metric` for early stopping, we also need to specify `stopping_metric=\"custom\"`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "2b4650c3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xgboost Model Build progress: |██████████████████████████████████████████████████| (done) 100%\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style='margin: 1em 0 1em 0;'>Model Details\n",
       "=============\n",
       "H2OXGBoostEstimator : XGBoost\n",
       "Model Key: XGBoost_model_python_1666962310297_567\n",
       "</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-11.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-11 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-11 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-11 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-11 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-11 .h2o-table th,\n",
       "#h2o-table-11 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-11 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-11\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Model Summary: </caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>number_of_trees</th></tr></thead>\n",
       "    <tbody><tr><td></td>\n",
       "<td>22.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'><pre style='margin: 1em 0 1em 0;'>ModelMetricsBinomial: xgboost\n",
       "** Reported on train data. **\n",
       "\n",
       "MSE: 0.03475774769265435\n",
       "RMSE: 0.18643429859511995\n",
       "LogLoss: 0.12901406890397824\n",
       "Mean Per-Class Error: 0.04387881533186782\n",
       "AUC: 0.9893340459497513\n",
       "AUCPR: 0.988704085388505\n",
       "Gini: 0.9786680918995025</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-12.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-12 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-12 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-12 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-12 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-12 .h2o-table th,\n",
       "#h2o-table-12 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-12 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-12\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.5101676438305829</caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>0</th>\n",
       "<th>1</th>\n",
       "<th>Error</th>\n",
       "<th>Rate</th></tr></thead>\n",
       "    <tbody><tr><td>0</td>\n",
       "<td>35155.0</td>\n",
       "<td>2174.0</td>\n",
       "<td>0.0582</td>\n",
       "<td> (2174.0/37329.0)</td></tr>\n",
       "<tr><td>1</td>\n",
       "<td>1112.0</td>\n",
       "<td>36559.0</td>\n",
       "<td>0.0295</td>\n",
       "<td> (1112.0/37671.0)</td></tr>\n",
       "<tr><td>Total</td>\n",
       "<td>36267.0</td>\n",
       "<td>38733.0</td>\n",
       "<td>0.0438</td>\n",
       "<td> (3286.0/75000.0)</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-13.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-13 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-13 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-13 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-13 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-13 .h2o-table th,\n",
       "#h2o-table-13 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-13 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-13\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Maximum Metrics: Maximum metrics at their respective thresholds</caption>\n",
       "    <thead><tr><th>metric</th>\n",
       "<th>threshold</th>\n",
       "<th>value</th>\n",
       "<th>idx</th></tr></thead>\n",
       "    <tbody><tr><td>max f1</td>\n",
       "<td>0.5101676</td>\n",
       "<td>0.9569918</td>\n",
       "<td>214.0</td></tr>\n",
       "<tr><td>max f2</td>\n",
       "<td>0.3082806</td>\n",
       "<td>0.9715081</td>\n",
       "<td>263.0</td></tr>\n",
       "<tr><td>max f0point5</td>\n",
       "<td>0.7532607</td>\n",
       "<td>0.9592084</td>\n",
       "<td>147.0</td></tr>\n",
       "<tr><td>max accuracy</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9564</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max precision</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max recall</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max specificity</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max absolute_mcc</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9129277</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max min_per_class_accuracy</td>\n",
       "<td>0.6239853</td>\n",
       "<td>0.9548073</td>\n",
       "<td>184.0</td></tr>\n",
       "<tr><td>max mean_per_class_accuracy</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9563597</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max tns</td>\n",
       "<td>0.9974565</td>\n",
       "<td>37329.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fns</td>\n",
       "<td>0.9974565</td>\n",
       "<td>37460.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fps</td>\n",
       "<td>0.0016028</td>\n",
       "<td>37329.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tps</td>\n",
       "<td>0.0016028</td>\n",
       "<td>37671.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tnr</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fnr</td>\n",
       "<td>0.9974565</td>\n",
       "<td>0.9943989</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fpr</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tpr</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-14.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-14 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-14 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-14 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-14 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-14 .h2o-table th,\n",
       "#h2o-table-14 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-14 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-14\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Gains/Lift Table: Avg response rate: 50.23 %, avg score: 50.22 %</caption>\n",
       "    <thead><tr><th>group</th>\n",
       "<th>cumulative_data_fraction</th>\n",
       "<th>lower_threshold</th>\n",
       "<th>lift</th>\n",
       "<th>cumulative_lift</th>\n",
       "<th>response_rate</th>\n",
       "<th>score</th>\n",
       "<th>cumulative_response_rate</th>\n",
       "<th>cumulative_score</th>\n",
       "<th>capture_rate</th>\n",
       "<th>cumulative_capture_rate</th>\n",
       "<th>gain</th>\n",
       "<th>cumulative_gain</th>\n",
       "<th>kolmogorov_smirnov</th></tr></thead>\n",
       "    <tbody><tr><td>1</td>\n",
       "<td>0.0101733</td>\n",
       "<td>0.9961097</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967516</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967516</td>\n",
       "<td>0.0202543</td>\n",
       "<td>0.0202543</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0202543</td></tr>\n",
       "<tr><td>2</td>\n",
       "<td>0.0200267</td>\n",
       "<td>0.9957153</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9958894</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9963274</td>\n",
       "<td>0.0196172</td>\n",
       "<td>0.0398715</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0398715</td></tr>\n",
       "<tr><td>3</td>\n",
       "<td>0.03068</td>\n",
       "<td>0.9953964</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9955495</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9960573</td>\n",
       "<td>0.0212099</td>\n",
       "<td>0.0610815</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0610815</td></tr>\n",
       "<tr><td>4</td>\n",
       "<td>0.0401067</td>\n",
       "<td>0.9951944</td>\n",
       "<td>1.9881054</td>\n",
       "<td>1.9902595</td>\n",
       "<td>0.9985856</td>\n",
       "<td>0.9952708</td>\n",
       "<td>0.9996676</td>\n",
       "<td>0.9958724</td>\n",
       "<td>0.0187412</td>\n",
       "<td>0.0798227</td>\n",
       "<td>98.8105385</td>\n",
       "<td>99.0259523</td>\n",
       "<td>0.0797959</td></tr>\n",
       "<tr><td>5</td>\n",
       "<td>0.0517733</td>\n",
       "<td>0.9949297</td>\n",
       "<td>1.9863707</td>\n",
       "<td>1.9893832</td>\n",
       "<td>0.9977143</td>\n",
       "<td>0.9950084</td>\n",
       "<td>0.9992274</td>\n",
       "<td>0.9956777</td>\n",
       "<td>0.0231743</td>\n",
       "<td>0.1029970</td>\n",
       "<td>98.6370721</td>\n",
       "<td>98.9383216</td>\n",
       "<td>0.1029166</td></tr>\n",
       "<tr><td>6</td>\n",
       "<td>0.1000133</td>\n",
       "<td>0.9933097</td>\n",
       "<td>1.9881700</td>\n",
       "<td>1.9887980</td>\n",
       "<td>0.9986180</td>\n",
       "<td>0.9941657</td>\n",
       "<td>0.9989335</td>\n",
       "<td>0.9949484</td>\n",
       "<td>0.0959093</td>\n",
       "<td>0.1989063</td>\n",
       "<td>98.8169987</td>\n",
       "<td>98.8798032</td>\n",
       "<td>0.1986920</td></tr>\n",
       "<tr><td>7</td>\n",
       "<td>0.15008</td>\n",
       "<td>0.9904984</td>\n",
       "<td>1.9840287</td>\n",
       "<td>1.9872070</td>\n",
       "<td>0.9965379</td>\n",
       "<td>0.9920577</td>\n",
       "<td>0.9981343</td>\n",
       "<td>0.9939841</td>\n",
       "<td>0.0993337</td>\n",
       "<td>0.2982400</td>\n",
       "<td>98.4028728</td>\n",
       "<td>98.7206993</td>\n",
       "<td>0.2976775</td></tr>\n",
       "<tr><td>8</td>\n",
       "<td>0.2000133</td>\n",
       "<td>0.9849605</td>\n",
       "<td>1.9824155</td>\n",
       "<td>1.9860108</td>\n",
       "<td>0.9957276</td>\n",
       "<td>0.9879125</td>\n",
       "<td>0.9975335</td>\n",
       "<td>0.9924683</td>\n",
       "<td>0.0989886</td>\n",
       "<td>0.3972286</td>\n",
       "<td>98.2415459</td>\n",
       "<td>98.6010786</td>\n",
       "<td>0.3962375</td></tr>\n",
       "<tr><td>9</td>\n",
       "<td>0.30004</td>\n",
       "<td>0.9561046</td>\n",
       "<td>1.9673021</td>\n",
       "<td>1.9797737</td>\n",
       "<td>0.9881365</td>\n",
       "<td>0.9740387</td>\n",
       "<td>0.9944007</td>\n",
       "<td>0.9863243</td>\n",
       "<td>0.1967827</td>\n",
       "<td>0.5940113</td>\n",
       "<td>96.7302096</td>\n",
       "<td>97.9773725</td>\n",
       "<td>0.5906359</td></tr>\n",
       "<tr><td>10</td>\n",
       "<td>0.4000133</td>\n",
       "<td>0.8770613</td>\n",
       "<td>1.9123254</td>\n",
       "<td>1.9629167</td>\n",
       "<td>0.9605228</td>\n",
       "<td>0.9210076</td>\n",
       "<td>0.9859338</td>\n",
       "<td>0.9700000</td>\n",
       "<td>0.1911815</td>\n",
       "<td>0.7851929</td>\n",
       "<td>91.2325408</td>\n",
       "<td>96.2916704</td>\n",
       "<td>0.7738880</td></tr>\n",
       "<tr><td>11</td>\n",
       "<td>0.5000133</td>\n",
       "<td>0.6383476</td>\n",
       "<td>1.6763558</td>\n",
       "<td>1.9056061</td>\n",
       "<td>0.842</td>\n",
       "<td>0.8051186</td>\n",
       "<td>0.9571478</td>\n",
       "<td>0.9370246</td>\n",
       "<td>0.1676356</td>\n",
       "<td>0.9528284</td>\n",
       "<td>67.6355817</td>\n",
       "<td>90.5606055</td>\n",
       "<td>0.9097788</td></tr>\n",
       "<tr><td>12</td>\n",
       "<td>0.6</td>\n",
       "<td>0.0781245</td>\n",
       "<td>0.4261140</td>\n",
       "<td>1.6590569</td>\n",
       "<td>0.2140285</td>\n",
       "<td>0.2876185</td>\n",
       "<td>0.8333111</td>\n",
       "<td>0.8288047</td>\n",
       "<td>0.0426057</td>\n",
       "<td>0.9954342</td>\n",
       "<td>-57.3886006</td>\n",
       "<td>65.9056923</td>\n",
       "<td>0.7944912</td></tr>\n",
       "<tr><td>13</td>\n",
       "<td>0.7000133</td>\n",
       "<td>0.0128222</td>\n",
       "<td>0.0238879</td>\n",
       "<td>1.4254346</td>\n",
       "<td>0.0119984</td>\n",
       "<td>0.0339001</td>\n",
       "<td>0.7159673</td>\n",
       "<td>0.7152339</td>\n",
       "<td>0.0023891</td>\n",
       "<td>0.9978233</td>\n",
       "<td>-97.6112128</td>\n",
       "<td>42.5434648</td>\n",
       "<td>0.5983483</td></tr>\n",
       "<tr><td>14</td>\n",
       "<td>0.80008</td>\n",
       "<td>0.0057930</td>\n",
       "<td>0.0122028</td>\n",
       "<td>1.2486806</td>\n",
       "<td>0.0061292</td>\n",
       "<td>0.0084165</td>\n",
       "<td>0.6271873</td>\n",
       "<td>0.6268317</td>\n",
       "<td>0.0012211</td>\n",
       "<td>0.9990444</td>\n",
       "<td>-98.7797151</td>\n",
       "<td>24.8680579</td>\n",
       "<td>0.3997516</td></tr>\n",
       "<tr><td>15</td>\n",
       "<td>0.9</td>\n",
       "<td>0.0034428</td>\n",
       "<td>0.0045164</td>\n",
       "<td>1.1105507</td>\n",
       "<td>0.0022685</td>\n",
       "<td>0.0045299</td>\n",
       "<td>0.5578074</td>\n",
       "<td>0.5577424</td>\n",
       "<td>0.0004513</td>\n",
       "<td>0.9994956</td>\n",
       "<td>-99.5483632</td>\n",
       "<td>11.0550704</td>\n",
       "<td>0.1999028</td></tr>\n",
       "<tr><td>16</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0001672</td>\n",
       "<td>0.0050437</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0025333</td>\n",
       "<td>0.0026268</td>\n",
       "<td>0.50228</td>\n",
       "<td>0.5022308</td>\n",
       "<td>0.0005044</td>\n",
       "<td>1.0</td>\n",
       "<td>-99.4956332</td>\n",
       "<td>0.0</td>\n",
       "<td>0.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div></div>\n",
       "<div style='margin: 1em 0 1em 0;'><pre style='margin: 1em 0 1em 0;'>ModelMetricsBinomial: xgboost\n",
       "** Reported on validation data. **\n",
       "\n",
       "MSE: 0.038600168581279404\n",
       "RMSE: 0.1964692560714765\n",
       "LogLoss: 0.14264211397729937\n",
       "Mean Per-Class Error: 0.04874265748103474\n",
       "AUC: 0.9860367984758349\n",
       "AUCPR: 0.9841777019906762\n",
       "Gini: 0.9720735969516698</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-15.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-15 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-15 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-15 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-15 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-15 .h2o-table th,\n",
       "#h2o-table-15 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-15 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-15\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.4899955771186135</caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>0</th>\n",
       "<th>1</th>\n",
       "<th>Error</th>\n",
       "<th>Rate</th></tr></thead>\n",
       "    <tbody><tr><td>0</td>\n",
       "<td>11844.0</td>\n",
       "<td>840.0</td>\n",
       "<td>0.0662</td>\n",
       "<td> (840.0/12684.0)</td></tr>\n",
       "<tr><td>1</td>\n",
       "<td>385.0</td>\n",
       "<td>11931.0</td>\n",
       "<td>0.0313</td>\n",
       "<td> (385.0/12316.0)</td></tr>\n",
       "<tr><td>Total</td>\n",
       "<td>12229.0</td>\n",
       "<td>12771.0</td>\n",
       "<td>0.049</td>\n",
       "<td> (1225.0/25000.0)</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-16.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-16 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-16 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-16 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-16 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-16 .h2o-table th,\n",
       "#h2o-table-16 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-16 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-16\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Maximum Metrics: Maximum metrics at their respective thresholds</caption>\n",
       "    <thead><tr><th>metric</th>\n",
       "<th>threshold</th>\n",
       "<th>value</th>\n",
       "<th>idx</th></tr></thead>\n",
       "    <tbody><tr><td>max f1</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9511699</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max f2</td>\n",
       "<td>0.2381639</td>\n",
       "<td>0.9684011</td>\n",
       "<td>281.0</td></tr>\n",
       "<tr><td>max f0point5</td>\n",
       "<td>0.7637601</td>\n",
       "<td>0.9521032</td>\n",
       "<td>144.0</td></tr>\n",
       "<tr><td>max accuracy</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.951</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max precision</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max recall</td>\n",
       "<td>0.0018354</td>\n",
       "<td>1.0</td>\n",
       "<td>398.0</td></tr>\n",
       "<tr><td>max specificity</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max absolute_mcc</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9026291</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max min_per_class_accuracy</td>\n",
       "<td>0.6303638</td>\n",
       "<td>0.9494966</td>\n",
       "<td>182.0</td></tr>\n",
       "<tr><td>max mean_per_class_accuracy</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9512573</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max tns</td>\n",
       "<td>0.9974683</td>\n",
       "<td>12684.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fns</td>\n",
       "<td>0.9974683</td>\n",
       "<td>12244.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fps</td>\n",
       "<td>0.0013435</td>\n",
       "<td>12684.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tps</td>\n",
       "<td>0.0018354</td>\n",
       "<td>12316.0</td>\n",
       "<td>398.0</td></tr>\n",
       "<tr><td>max tnr</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fnr</td>\n",
       "<td>0.9974683</td>\n",
       "<td>0.9941539</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fpr</td>\n",
       "<td>0.0013435</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tpr</td>\n",
       "<td>0.0018354</td>\n",
       "<td>1.0</td>\n",
       "<td>398.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-17.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-17 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-17 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-17 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-17 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-17 .h2o-table th,\n",
       "#h2o-table-17 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-17 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-17\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Gains/Lift Table: Avg response rate: 49.26 %, avg score: 49.47 %</caption>\n",
       "    <thead><tr><th>group</th>\n",
       "<th>cumulative_data_fraction</th>\n",
       "<th>lower_threshold</th>\n",
       "<th>lift</th>\n",
       "<th>cumulative_lift</th>\n",
       "<th>response_rate</th>\n",
       "<th>score</th>\n",
       "<th>cumulative_response_rate</th>\n",
       "<th>cumulative_score</th>\n",
       "<th>capture_rate</th>\n",
       "<th>cumulative_capture_rate</th>\n",
       "<th>gain</th>\n",
       "<th>cumulative_gain</th>\n",
       "<th>kolmogorov_smirnov</th></tr></thead>\n",
       "    <tbody><tr><td>1</td>\n",
       "<td>0.01028</td>\n",
       "<td>0.9961244</td>\n",
       "<td>2.0298798</td>\n",
       "<td>2.0298798</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967313</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967313</td>\n",
       "<td>0.0208672</td>\n",
       "<td>0.0208672</td>\n",
       "<td>102.9879831</td>\n",
       "<td>102.9879831</td>\n",
       "<td>0.0208672</td></tr>\n",
       "<tr><td>2</td>\n",
       "<td>0.02032</td>\n",
       "<td>0.9958012</td>\n",
       "<td>2.0298798</td>\n",
       "<td>2.0298798</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9959175</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9963292</td>\n",
       "<td>0.0203800</td>\n",
       "<td>0.0412472</td>\n",
       "<td>102.9879831</td>\n",
       "<td>102.9879831</td>\n",
       "<td>0.0412472</td></tr>\n",
       "<tr><td>3</td>\n",
       "<td>0.03016</td>\n",
       "<td>0.9954615</td>\n",
       "<td>2.0216283</td>\n",
       "<td>2.0271877</td>\n",
       "<td>0.9959350</td>\n",
       "<td>0.9956102</td>\n",
       "<td>0.9986737</td>\n",
       "<td>0.9960946</td>\n",
       "<td>0.0198928</td>\n",
       "<td>0.0611400</td>\n",
       "<td>102.1628287</td>\n",
       "<td>102.7187683</td>\n",
       "<td>0.0610611</td></tr>\n",
       "<tr><td>4</td>\n",
       "<td>0.04108</td>\n",
       "<td>0.9951944</td>\n",
       "<td>2.0224444</td>\n",
       "<td>2.0259268</td>\n",
       "<td>0.9963370</td>\n",
       "<td>0.9952887</td>\n",
       "<td>0.9980526</td>\n",
       "<td>0.9958804</td>\n",
       "<td>0.0220851</td>\n",
       "<td>0.0832251</td>\n",
       "<td>102.2444374</td>\n",
       "<td>102.5926803</td>\n",
       "<td>0.0830674</td></tr>\n",
       "<tr><td>5</td>\n",
       "<td>0.05256</td>\n",
       "<td>0.9949297</td>\n",
       "<td>2.0157343</td>\n",
       "<td>2.0237006</td>\n",
       "<td>0.9930314</td>\n",
       "<td>0.9950085</td>\n",
       "<td>0.9969559</td>\n",
       "<td>0.9956900</td>\n",
       "<td>0.0231406</td>\n",
       "<td>0.1063657</td>\n",
       "<td>101.5734327</td>\n",
       "<td>102.3700593</td>\n",
       "<td>0.1060503</td></tr>\n",
       "<tr><td>6</td>\n",
       "<td>0.1</td>\n",
       "<td>0.9932859</td>\n",
       "<td>2.0196106</td>\n",
       "<td>2.0217603</td>\n",
       "<td>0.9949410</td>\n",
       "<td>0.9941552</td>\n",
       "<td>0.996</td>\n",
       "<td>0.9949619</td>\n",
       "<td>0.0958103</td>\n",
       "<td>0.2021760</td>\n",
       "<td>101.9610625</td>\n",
       "<td>102.1760312</td>\n",
       "<td>0.2013876</td></tr>\n",
       "<tr><td>7</td>\n",
       "<td>0.15</td>\n",
       "<td>0.9902694</td>\n",
       "<td>2.0233842</td>\n",
       "<td>2.0223016</td>\n",
       "<td>0.9968</td>\n",
       "<td>0.9919490</td>\n",
       "<td>0.9962667</td>\n",
       "<td>0.9939576</td>\n",
       "<td>0.1011692</td>\n",
       "<td>0.3033452</td>\n",
       "<td>102.3384216</td>\n",
       "<td>102.2301613</td>\n",
       "<td>0.3022415</td></tr>\n",
       "<tr><td>8</td>\n",
       "<td>0.2</td>\n",
       "<td>0.9840660</td>\n",
       "<td>2.0136408</td>\n",
       "<td>2.0201364</td>\n",
       "<td>0.992</td>\n",
       "<td>0.9873945</td>\n",
       "<td>0.9952</td>\n",
       "<td>0.9923168</td>\n",
       "<td>0.1006820</td>\n",
       "<td>0.4040273</td>\n",
       "<td>101.3640792</td>\n",
       "<td>102.0136408</td>\n",
       "<td>0.4021351</td></tr>\n",
       "<tr><td>9</td>\n",
       "<td>0.3</td>\n",
       "<td>0.9523897</td>\n",
       "<td>1.9852225</td>\n",
       "<td>2.0084984</td>\n",
       "<td>0.978</td>\n",
       "<td>0.9717051</td>\n",
       "<td>0.9894667</td>\n",
       "<td>0.9854462</td>\n",
       "<td>0.1985222</td>\n",
       "<td>0.6025495</td>\n",
       "<td>98.5222475</td>\n",
       "<td>100.8498430</td>\n",
       "<td>0.5963212</td></tr>\n",
       "<tr><td>10</td>\n",
       "<td>0.4</td>\n",
       "<td>0.8671057</td>\n",
       "<td>1.9235141</td>\n",
       "<td>1.9872524</td>\n",
       "<td>0.9476</td>\n",
       "<td>0.9142567</td>\n",
       "<td>0.979</td>\n",
       "<td>0.9676488</td>\n",
       "<td>0.1923514</td>\n",
       "<td>0.7949009</td>\n",
       "<td>92.3514128</td>\n",
       "<td>98.7252355</td>\n",
       "<td>0.7783447</td></tr>\n",
       "<tr><td>11</td>\n",
       "<td>0.5</td>\n",
       "<td>0.5822353</td>\n",
       "<td>1.6174082</td>\n",
       "<td>1.9132835</td>\n",
       "<td>0.7968</td>\n",
       "<td>0.7825059</td>\n",
       "<td>0.94256</td>\n",
       "<td>0.9306202</td>\n",
       "<td>0.1617408</td>\n",
       "<td>0.9566418</td>\n",
       "<td>61.7408249</td>\n",
       "<td>91.3283534</td>\n",
       "<td>0.9000350</td></tr>\n",
       "<tr><td>12</td>\n",
       "<td>0.6</td>\n",
       "<td>0.0667765</td>\n",
       "<td>0.3775576</td>\n",
       "<td>1.6573292</td>\n",
       "<td>0.186</td>\n",
       "<td>0.2480180</td>\n",
       "<td>0.8164667</td>\n",
       "<td>0.8168532</td>\n",
       "<td>0.0377558</td>\n",
       "<td>0.9943975</td>\n",
       "<td>-62.2442351</td>\n",
       "<td>65.7329219</td>\n",
       "<td>0.7773524</td></tr>\n",
       "<tr><td>13</td>\n",
       "<td>0.7</td>\n",
       "<td>0.0125771</td>\n",
       "<td>0.0186749</td>\n",
       "<td>1.4232357</td>\n",
       "<td>0.0092</td>\n",
       "<td>0.0304482</td>\n",
       "<td>0.7011429</td>\n",
       "<td>0.7045096</td>\n",
       "<td>0.0018675</td>\n",
       "<td>0.9962650</td>\n",
       "<td>-98.1325106</td>\n",
       "<td>42.3235744</td>\n",
       "<td>0.5839345</td></tr>\n",
       "<tr><td>14</td>\n",
       "<td>0.80048</td>\n",
       "<td>0.0057412</td>\n",
       "<td>0.0185857</td>\n",
       "<td>1.2469175</td>\n",
       "<td>0.0091561</td>\n",
       "<td>0.0081574</td>\n",
       "<td>0.6142814</td>\n",
       "<td>0.6171002</td>\n",
       "<td>0.0018675</td>\n",
       "<td>0.9981325</td>\n",
       "<td>-98.1414317</td>\n",
       "<td>24.6917488</td>\n",
       "<td>0.3895705</td></tr>\n",
       "<tr><td>15</td>\n",
       "<td>0.9014</td>\n",
       "<td>0.0034592</td>\n",
       "<td>0.0104592</td>\n",
       "<td>1.1084846</td>\n",
       "<td>0.0051526</td>\n",
       "<td>0.0045211</td>\n",
       "<td>0.5460839</td>\n",
       "<td>0.5485164</td>\n",
       "<td>0.0010555</td>\n",
       "<td>0.9991880</td>\n",
       "<td>-98.9540849</td>\n",
       "<td>10.8484633</td>\n",
       "<td>0.1927390</td></tr>\n",
       "<tr><td>16</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0002609</td>\n",
       "<td>0.0082348</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0040568</td>\n",
       "<td>0.0026446</td>\n",
       "<td>0.49264</td>\n",
       "<td>0.4946934</td>\n",
       "<td>0.0008120</td>\n",
       "<td>1.0</td>\n",
       "<td>-99.1765193</td>\n",
       "<td>0.0</td>\n",
       "<td>0.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div></div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-18.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-18 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-18 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-18 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-18 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-18 .h2o-table th,\n",
       "#h2o-table-18 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-18 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-18\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Scoring History: </caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>timestamp</th>\n",
       "<th>duration</th>\n",
       "<th>number_of_trees</th>\n",
       "<th>training_rmse</th>\n",
       "<th>training_logloss</th>\n",
       "<th>training_auc</th>\n",
       "<th>training_pr_auc</th>\n",
       "<th>training_lift</th>\n",
       "<th>training_classification_error</th>\n",
       "<th>training_custom</th>\n",
       "<th>validation_rmse</th>\n",
       "<th>validation_logloss</th>\n",
       "<th>validation_auc</th>\n",
       "<th>validation_pr_auc</th>\n",
       "<th>validation_lift</th>\n",
       "<th>validation_classification_error</th>\n",
       "<th>validation_custom</th></tr></thead>\n",
       "    <tbody><tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.025 sec</td>\n",
       "<td>0.0</td>\n",
       "<td>0.5</td>\n",
       "<td>0.6931472</td>\n",
       "<td>0.5</td>\n",
       "<td>0.5022800</td>\n",
       "<td>1.0</td>\n",
       "<td>0.49772</td>\n",
       "<td>0.693147</td>\n",
       "<td>0.5</td>\n",
       "<td>0.6931472</td>\n",
       "<td>0.5</td>\n",
       "<td>0.4926400</td>\n",
       "<td>1.0</td>\n",
       "<td>0.50736</td>\n",
       "<td>0.693147</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.139 sec</td>\n",
       "<td>1.0</td>\n",
       "<td>0.3996010</td>\n",
       "<td>0.5084556</td>\n",
       "<td>0.9599605</td>\n",
       "<td>0.9528824</td>\n",
       "<td>1.9699443</td>\n",
       "<td>0.08508</td>\n",
       "<td>0.508456</td>\n",
       "<td>0.4002939</td>\n",
       "<td>0.5095766</td>\n",
       "<td>0.9584754</td>\n",
       "<td>0.9501212</td>\n",
       "<td>2.0073256</td>\n",
       "<td>0.08816</td>\n",
       "<td>0.509577</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.179 sec</td>\n",
       "<td>2.0</td>\n",
       "<td>0.3370778</td>\n",
       "<td>0.4032048</td>\n",
       "<td>0.9693358</td>\n",
       "<td>0.9638432</td>\n",
       "<td>1.9749781</td>\n",
       "<td>0.0786933</td>\n",
       "<td>0.403205</td>\n",
       "<td>0.3378726</td>\n",
       "<td>0.4043148</td>\n",
       "<td>0.9689642</td>\n",
       "<td>0.9627452</td>\n",
       "<td>2.0180782</td>\n",
       "<td>0.08072</td>\n",
       "<td>0.404315</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.228 sec</td>\n",
       "<td>3.0</td>\n",
       "<td>0.2957135</td>\n",
       "<td>0.3335651</td>\n",
       "<td>0.9735764</td>\n",
       "<td>0.9692971</td>\n",
       "<td>1.9823583</td>\n",
       "<td>0.07124</td>\n",
       "<td>0.333565</td>\n",
       "<td>0.2972804</td>\n",
       "<td>0.3355690</td>\n",
       "<td>0.9727585</td>\n",
       "<td>0.9672690</td>\n",
       "<td>2.0164813</td>\n",
       "<td>0.07328</td>\n",
       "<td>0.335569</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.280 sec</td>\n",
       "<td>4.0</td>\n",
       "<td>0.2690565</td>\n",
       "<td>0.2867050</td>\n",
       "<td>0.9769386</td>\n",
       "<td>0.9736661</td>\n",
       "<td>1.9824404</td>\n",
       "<td>0.0687733</td>\n",
       "<td>0.286705</td>\n",
       "<td>0.2711930</td>\n",
       "<td>0.2894303</td>\n",
       "<td>0.9753680</td>\n",
       "<td>0.9709177</td>\n",
       "<td>2.0166126</td>\n",
       "<td>0.07028</td>\n",
       "<td>0.28943</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.356 sec</td>\n",
       "<td>5.0</td>\n",
       "<td>0.2529770</td>\n",
       "<td>0.2548994</td>\n",
       "<td>0.9782349</td>\n",
       "<td>0.9753981</td>\n",
       "<td>1.9830013</td>\n",
       "<td>0.06936</td>\n",
       "<td>0.254899</td>\n",
       "<td>0.2550437</td>\n",
       "<td>0.2575265</td>\n",
       "<td>0.9769549</td>\n",
       "<td>0.9724197</td>\n",
       "<td>2.0204531</td>\n",
       "<td>0.06956</td>\n",
       "<td>0.257526</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.434 sec</td>\n",
       "<td>6.0</td>\n",
       "<td>0.2430550</td>\n",
       "<td>0.2328859</td>\n",
       "<td>0.9792536</td>\n",
       "<td>0.9767333</td>\n",
       "<td>1.9826945</td>\n",
       "<td>0.06864</td>\n",
       "<td>0.232886</td>\n",
       "<td>0.2452571</td>\n",
       "<td>0.2357295</td>\n",
       "<td>0.9778085</td>\n",
       "<td>0.9737537</td>\n",
       "<td>2.0198804</td>\n",
       "<td>0.06872</td>\n",
       "<td>0.235729</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.511 sec</td>\n",
       "<td>7.0</td>\n",
       "<td>0.2366703</td>\n",
       "<td>0.2173043</td>\n",
       "<td>0.9801005</td>\n",
       "<td>0.9779518</td>\n",
       "<td>1.9840125</td>\n",
       "<td>0.0655333</td>\n",
       "<td>0.217304</td>\n",
       "<td>0.2388924</td>\n",
       "<td>0.2202064</td>\n",
       "<td>0.9787945</td>\n",
       "<td>0.9751087</td>\n",
       "<td>2.0189075</td>\n",
       "<td>0.06696</td>\n",
       "<td>0.220206</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:30</td>\n",
       "<td> 0.587 sec</td>\n",
       "<td>8.0</td>\n",
       "<td>0.2274282</td>\n",
       "<td>0.2004642</td>\n",
       "<td>0.9816894</td>\n",
       "<td>0.9795849</td>\n",
       "<td>1.9883425</td>\n",
       "<td>0.06172</td>\n",
       "<td>0.200464</td>\n",
       "<td>0.2302864</td>\n",
       "<td>0.2040544</td>\n",
       "<td>0.9803330</td>\n",
       "<td>0.9772714</td>\n",
       "<td>2.0219506</td>\n",
       "<td>0.06516</td>\n",
       "<td>0.204054</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:31</td>\n",
       "<td> 0.676 sec</td>\n",
       "<td>9.0</td>\n",
       "<td>0.2235048</td>\n",
       "<td>0.1908878</td>\n",
       "<td>0.9823444</td>\n",
       "<td>0.9805219</td>\n",
       "<td>1.9883948</td>\n",
       "<td>0.0608</td>\n",
       "<td>0.190888</td>\n",
       "<td>0.2267565</td>\n",
       "<td>0.1951048</td>\n",
       "<td>0.9808128</td>\n",
       "<td>0.9779093</td>\n",
       "<td>2.0222773</td>\n",
       "<td>0.06432</td>\n",
       "<td>0.195105</td></tr>\n",
       "<tr><td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:31</td>\n",
       "<td> 1.058 sec</td>\n",
       "<td>13.0</td>\n",
       "<td>0.2093299</td>\n",
       "<td>0.1618817</td>\n",
       "<td>0.9849208</td>\n",
       "<td>0.9835753</td>\n",
       "<td>1.9885342</td>\n",
       "<td>0.0549333</td>\n",
       "<td>0.161882</td>\n",
       "<td>0.2140217</td>\n",
       "<td>0.1682975</td>\n",
       "<td>0.9829089</td>\n",
       "<td>0.9803703</td>\n",
       "<td>2.0227574</td>\n",
       "<td>0.0584</td>\n",
       "<td>0.168298</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:31</td>\n",
       "<td> 1.160 sec</td>\n",
       "<td>14.0</td>\n",
       "<td>0.2080116</td>\n",
       "<td>0.1587226</td>\n",
       "<td>0.9852452</td>\n",
       "<td>0.9839538</td>\n",
       "<td>1.9883086</td>\n",
       "<td>0.0544</td>\n",
       "<td>0.158723</td>\n",
       "<td>0.2129618</td>\n",
       "<td>0.1656925</td>\n",
       "<td>0.9831326</td>\n",
       "<td>0.9806934</td>\n",
       "<td>2.0222773</td>\n",
       "<td>0.05852</td>\n",
       "<td>0.165693</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:31</td>\n",
       "<td> 1.246 sec</td>\n",
       "<td>15.0</td>\n",
       "<td>0.2065626</td>\n",
       "<td>0.1556114</td>\n",
       "<td>0.9856595</td>\n",
       "<td>0.9846933</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0538533</td>\n",
       "<td>0.155611</td>\n",
       "<td>0.2118815</td>\n",
       "<td>0.1633915</td>\n",
       "<td>0.9833377</td>\n",
       "<td>0.9810162</td>\n",
       "<td>2.0227823</td>\n",
       "<td>0.05792</td>\n",
       "<td>0.163391</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:31</td>\n",
       "<td> 1.345 sec</td>\n",
       "<td>16.0</td>\n",
       "<td>0.2036096</td>\n",
       "<td>0.1517939</td>\n",
       "<td>0.9863474</td>\n",
       "<td>0.9853808</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0520267</td>\n",
       "<td>0.151794</td>\n",
       "<td>0.2099121</td>\n",
       "<td>0.1607776</td>\n",
       "<td>0.9837030</td>\n",
       "<td>0.9814828</td>\n",
       "<td>2.0217603</td>\n",
       "<td>0.0566</td>\n",
       "<td>0.160778</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:31</td>\n",
       "<td> 1.462 sec</td>\n",
       "<td>17.0</td>\n",
       "<td>0.2001280</td>\n",
       "<td>0.1470822</td>\n",
       "<td>0.9868868</td>\n",
       "<td>0.9859104</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0503733</td>\n",
       "<td>0.147082</td>\n",
       "<td>0.2068741</td>\n",
       "<td>0.1565508</td>\n",
       "<td>0.9842040</td>\n",
       "<td>0.9821555</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.05524</td>\n",
       "<td>0.156551</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:31</td>\n",
       "<td> 1.566 sec</td>\n",
       "<td>18.0</td>\n",
       "<td>0.1952364</td>\n",
       "<td>0.1409688</td>\n",
       "<td>0.9877667</td>\n",
       "<td>0.9868323</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0479067</td>\n",
       "<td>0.140969</td>\n",
       "<td>0.2029770</td>\n",
       "<td>0.1516665</td>\n",
       "<td>0.9848696</td>\n",
       "<td>0.9829746</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.05272</td>\n",
       "<td>0.151666</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:32</td>\n",
       "<td> 1.684 sec</td>\n",
       "<td>19.0</td>\n",
       "<td>0.1943690</td>\n",
       "<td>0.1391751</td>\n",
       "<td>0.9880068</td>\n",
       "<td>0.9871283</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0476667</td>\n",
       "<td>0.139175</td>\n",
       "<td>0.2025051</td>\n",
       "<td>0.1505835</td>\n",
       "<td>0.9849703</td>\n",
       "<td>0.9829211</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.05232</td>\n",
       "<td>0.150583</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:32</td>\n",
       "<td> 1.802 sec</td>\n",
       "<td>20.0</td>\n",
       "<td>0.1919489</td>\n",
       "<td>0.1361213</td>\n",
       "<td>0.9884007</td>\n",
       "<td>0.9875789</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0466933</td>\n",
       "<td>0.136121</td>\n",
       "<td>0.2009170</td>\n",
       "<td>0.1484152</td>\n",
       "<td>0.9852986</td>\n",
       "<td>0.9834889</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.05108</td>\n",
       "<td>0.148415</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:32</td>\n",
       "<td> 1.916 sec</td>\n",
       "<td>21.0</td>\n",
       "<td>0.1886257</td>\n",
       "<td>0.1319356</td>\n",
       "<td>0.9889611</td>\n",
       "<td>0.9882193</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0444533</td>\n",
       "<td>0.131936</td>\n",
       "<td>0.1983105</td>\n",
       "<td>0.1448947</td>\n",
       "<td>0.9857546</td>\n",
       "<td>0.9838308</td>\n",
       "<td>2.0217603</td>\n",
       "<td>0.0496</td>\n",
       "<td>0.144895</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:32</td>\n",
       "<td> 2.057 sec</td>\n",
       "<td>22.0</td>\n",
       "<td>0.1864343</td>\n",
       "<td>0.1290141</td>\n",
       "<td>0.9893340</td>\n",
       "<td>0.9887041</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0438133</td>\n",
       "<td>0.129014</td>\n",
       "<td>0.1964693</td>\n",
       "<td>0.1426421</td>\n",
       "<td>0.9860368</td>\n",
       "<td>0.9841777</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.049</td>\n",
       "<td>0.142642</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "<pre style='font-size: smaller; margin-bottom: 1em;'>[23 rows x 18 columns]</pre></div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-19.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-19 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-19 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-19 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-19 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-19 .h2o-table th,\n",
       "#h2o-table-19 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-19 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-19\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Variable Importances: </caption>\n",
       "    <thead><tr><th>variable</th>\n",
       "<th>relative_importance</th>\n",
       "<th>scaled_importance</th>\n",
       "<th>percentage</th></tr></thead>\n",
       "    <tbody><tr><td>C11</td>\n",
       "<td>70178.75</td>\n",
       "<td>1.0</td>\n",
       "<td>0.4261358</td></tr>\n",
       "<tr><td>C2</td>\n",
       "<td>35732.6640625</td>\n",
       "<td>0.5091664</td>\n",
       "<td>0.2169741</td></tr>\n",
       "<tr><td>C13</td>\n",
       "<td>21608.4863281</td>\n",
       "<td>0.3079064</td>\n",
       "<td>0.1312100</td></tr>\n",
       "<tr><td>C8</td>\n",
       "<td>15654.5498047</td>\n",
       "<td>0.2230668</td>\n",
       "<td>0.0950568</td></tr>\n",
       "<tr><td>C3</td>\n",
       "<td>12402.2607422</td>\n",
       "<td>0.1767239</td>\n",
       "<td>0.0753084</td></tr>\n",
       "<tr><td>C19</td>\n",
       "<td>5739.7641602</td>\n",
       "<td>0.0817878</td>\n",
       "<td>0.0348527</td></tr>\n",
       "<tr><td>C5</td>\n",
       "<td>2835.9191895</td>\n",
       "<td>0.0404099</td>\n",
       "<td>0.0172201</td></tr>\n",
       "<tr><td>C6</td>\n",
       "<td>88.9352493</td>\n",
       "<td>0.0012673</td>\n",
       "<td>0.0005400</td></tr>\n",
       "<tr><td>C12</td>\n",
       "<td>51.9876480</td>\n",
       "<td>0.0007408</td>\n",
       "<td>0.0003157</td></tr>\n",
       "<tr><td>C17</td>\n",
       "<td>49.1711044</td>\n",
       "<td>0.0007007</td>\n",
       "<td>0.0002986</td></tr>\n",
       "<tr><td>C7</td>\n",
       "<td>48.0521317</td>\n",
       "<td>0.0006847</td>\n",
       "<td>0.0002918</td></tr>\n",
       "<tr><td>C14</td>\n",
       "<td>46.5472183</td>\n",
       "<td>0.0006633</td>\n",
       "<td>0.0002826</td></tr>\n",
       "<tr><td>C10</td>\n",
       "<td>43.8788605</td>\n",
       "<td>0.0006252</td>\n",
       "<td>0.0002664</td></tr>\n",
       "<tr><td>C4</td>\n",
       "<td>42.4346542</td>\n",
       "<td>0.0006047</td>\n",
       "<td>0.0002577</td></tr>\n",
       "<tr><td>C18</td>\n",
       "<td>39.3952980</td>\n",
       "<td>0.0005614</td>\n",
       "<td>0.0002392</td></tr>\n",
       "<tr><td>C1</td>\n",
       "<td>32.5539932</td>\n",
       "<td>0.0004639</td>\n",
       "<td>0.0001977</td></tr>\n",
       "<tr><td>C15</td>\n",
       "<td>25.9528160</td>\n",
       "<td>0.0003698</td>\n",
       "<td>0.0001576</td></tr>\n",
       "<tr><td>C20</td>\n",
       "<td>23.1192322</td>\n",
       "<td>0.0003294</td>\n",
       "<td>0.0001404</td></tr>\n",
       "<tr><td>C16</td>\n",
       "<td>22.9983807</td>\n",
       "<td>0.0003277</td>\n",
       "<td>0.0001396</td></tr>\n",
       "<tr><td>C9</td>\n",
       "<td>18.9113159</td>\n",
       "<td>0.0002695</td>\n",
       "<td>0.0001148</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div><pre style=\"font-size: smaller; margin: 1em 0 0 0;\">\n",
       "\n",
       "[tips]\n",
       "Use `model.explain()` to inspect the model.\n",
       "--\n",
       "Use `h2o.display.toggle_user_tips()` to switch on/off this section.</pre>"
      ],
      "text/plain": [
       "Model Details\n",
       "=============\n",
       "H2OXGBoostEstimator : XGBoost\n",
       "Model Key: XGBoost_model_python_1666962310297_567\n",
       "\n",
       "\n",
       "Model Summary: \n",
       "    number_of_trees\n",
       "--  -----------------\n",
       "    22\n",
       "\n",
       "ModelMetricsBinomial: xgboost\n",
       "** Reported on train data. **\n",
       "\n",
       "MSE: 0.03475774769265435\n",
       "RMSE: 0.18643429859511995\n",
       "LogLoss: 0.12901406890397824\n",
       "Mean Per-Class Error: 0.04387881533186782\n",
       "AUC: 0.9893340459497513\n",
       "AUCPR: 0.988704085388505\n",
       "Gini: 0.9786680918995025\n",
       "\n",
       "Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.5101676438305829\n",
       "       0      1      Error    Rate\n",
       "-----  -----  -----  -------  ----------------\n",
       "0      35155  2174   0.0582   (2174.0/37329.0)\n",
       "1      1112   36559  0.0295   (1112.0/37671.0)\n",
       "Total  36267  38733  0.0438   (3286.0/75000.0)\n",
       "\n",
       "Maximum Metrics: Maximum metrics at their respective thresholds\n",
       "metric                       threshold    value     idx\n",
       "---------------------------  -----------  --------  -----\n",
       "max f1                       0.510168     0.956992  214\n",
       "max f2                       0.308281     0.971508  263\n",
       "max f0point5                 0.753261     0.959208  147\n",
       "max accuracy                 0.556171     0.9564    201\n",
       "max precision                0.997456     1         0\n",
       "max recall                   0.00160282   1         399\n",
       "max specificity              0.997456     1         0\n",
       "max absolute_mcc             0.556171     0.912928  201\n",
       "max min_per_class_accuracy   0.623985     0.954807  184\n",
       "max mean_per_class_accuracy  0.556171     0.95636   201\n",
       "max tns                      0.997456     37329     0\n",
       "max fns                      0.997456     37460     0\n",
       "max fps                      0.00160282   37329     399\n",
       "max tps                      0.00160282   37671     399\n",
       "max tnr                      0.997456     1         0\n",
       "max fnr                      0.997456     0.994399  0\n",
       "max fpr                      0.00160282   1         399\n",
       "max tpr                      0.00160282   1         399\n",
       "\n",
       "Gains/Lift Table: Avg response rate: 50.23 %, avg score: 50.22 %\n",
       "group    cumulative_data_fraction    lower_threshold    lift        cumulative_lift    response_rate    score       cumulative_response_rate    cumulative_score    capture_rate    cumulative_capture_rate    gain      cumulative_gain    kolmogorov_smirnov\n",
       "-------  --------------------------  -----------------  ----------  -----------------  ---------------  ----------  --------------------------  ------------------  --------------  -------------------------  --------  -----------------  --------------------\n",
       "1        0.0101733                   0.99611            1.99092     1.99092            1                0.996752    1                           0.996752            0.0202543       0.0202543                  99.0921   99.0921            0.0202543\n",
       "2        0.0200267                   0.995715           1.99092     1.99092            1                0.995889    1                           0.996327            0.0196172       0.0398715                  99.0921   99.0921            0.0398715\n",
       "3        0.03068                     0.995396           1.99092     1.99092            1                0.995549    1                           0.996057            0.0212099       0.0610815                  99.0921   99.0921            0.0610815\n",
       "4        0.0401067                   0.995194           1.98811     1.99026            0.998586         0.995271    0.999668                    0.995872            0.0187412       0.0798227                  98.8105   99.026             0.0797959\n",
       "5        0.0517733                   0.99493            1.98637     1.98938            0.997714         0.995008    0.999227                    0.995678            0.0231743       0.102997                   98.6371   98.9383            0.102917\n",
       "6        0.100013                    0.99331            1.98817     1.9888             0.998618         0.994166    0.998933                    0.994948            0.0959093       0.198906                   98.817    98.8798            0.198692\n",
       "7        0.15008                     0.990498           1.98403     1.98721            0.996538         0.992058    0.998134                    0.993984            0.0993337       0.29824                    98.4029   98.7207            0.297677\n",
       "8        0.200013                    0.98496            1.98242     1.98601            0.995728         0.987913    0.997533                    0.992468            0.0989886       0.397229                   98.2415   98.6011            0.396237\n",
       "9        0.30004                     0.956105           1.9673      1.97977            0.988136         0.974039    0.994401                    0.986324            0.196783        0.594011                   96.7302   97.9774            0.590636\n",
       "10       0.400013                    0.877061           1.91233     1.96292            0.960523         0.921008    0.985934                    0.97                0.191182        0.785193                   91.2325   96.2917            0.773888\n",
       "11       0.500013                    0.638348           1.67636     1.90561            0.842            0.805119    0.957148                    0.937025            0.167636        0.952828                   67.6356   90.5606            0.909779\n",
       "12       0.6                         0.0781245          0.426114    1.65906            0.214029         0.287618    0.833311                    0.828805            0.0426057       0.995434                   -57.3886  65.9057            0.794491\n",
       "13       0.700013                    0.0128222          0.0238879   1.42543            0.0119984        0.0339001   0.715967                    0.715234            0.00238911      0.997823                   -97.6112  42.5435            0.598348\n",
       "14       0.80008                     0.00579305         0.0122028   1.24868            0.00612925       0.00841651  0.627187                    0.626832            0.0012211       0.999044                   -98.7797  24.8681            0.399752\n",
       "15       0.9                         0.00344276         0.00451637  1.11055            0.00226848       0.00452987  0.557807                    0.557742            0.000451276     0.999496                   -99.5484  11.0551            0.199903\n",
       "16       1                           0.000167174        0.00504367  1                  0.00253333       0.00262682  0.50228                     0.502231            0.000504367     1                          -99.4956  0                  0\n",
       "\n",
       "ModelMetricsBinomial: xgboost\n",
       "** Reported on validation data. **\n",
       "\n",
       "MSE: 0.038600168581279404\n",
       "RMSE: 0.1964692560714765\n",
       "LogLoss: 0.14264211397729937\n",
       "Mean Per-Class Error: 0.04874265748103474\n",
       "AUC: 0.9860367984758349\n",
       "AUCPR: 0.9841777019906762\n",
       "Gini: 0.9720735969516698\n",
       "\n",
       "Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.4899955771186135\n",
       "       0      1      Error    Rate\n",
       "-----  -----  -----  -------  ----------------\n",
       "0      11844  840    0.0662   (840.0/12684.0)\n",
       "1      385    11931  0.0313   (385.0/12316.0)\n",
       "Total  12229  12771  0.049    (1225.0/25000.0)\n",
       "\n",
       "Maximum Metrics: Maximum metrics at their respective thresholds\n",
       "metric                       threshold    value     idx\n",
       "---------------------------  -----------  --------  -----\n",
       "max f1                       0.489996     0.95117   217\n",
       "max f2                       0.238164     0.968401  281\n",
       "max f0point5                 0.76376      0.952103  144\n",
       "max accuracy                 0.489996     0.951     217\n",
       "max precision                0.997468     1         0\n",
       "max recall                   0.00183535   1         398\n",
       "max specificity              0.997468     1         0\n",
       "max absolute_mcc             0.489996     0.902629  217\n",
       "max min_per_class_accuracy   0.630364     0.949497  182\n",
       "max mean_per_class_accuracy  0.489996     0.951257  217\n",
       "max tns                      0.997468     12684     0\n",
       "max fns                      0.997468     12244     0\n",
       "max fps                      0.00134346   12684     399\n",
       "max tps                      0.00183535   12316     398\n",
       "max tnr                      0.997468     1         0\n",
       "max fnr                      0.997468     0.994154  0\n",
       "max fpr                      0.00134346   1         399\n",
       "max tpr                      0.00183535   1         398\n",
       "\n",
       "Gains/Lift Table: Avg response rate: 49.26 %, avg score: 49.47 %\n",
       "group    cumulative_data_fraction    lower_threshold    lift        cumulative_lift    response_rate    score       cumulative_response_rate    cumulative_score    capture_rate    cumulative_capture_rate    gain      cumulative_gain    kolmogorov_smirnov\n",
       "-------  --------------------------  -----------------  ----------  -----------------  ---------------  ----------  --------------------------  ------------------  --------------  -------------------------  --------  -----------------  --------------------\n",
       "1        0.01028                     0.996124           2.02988     2.02988            1                0.996731    1                           0.996731            0.0208672       0.0208672                  102.988   102.988            0.0208672\n",
       "2        0.02032                     0.995801           2.02988     2.02988            1                0.995918    1                           0.996329            0.02038         0.0412472                  102.988   102.988            0.0412472\n",
       "3        0.03016                     0.995462           2.02163     2.02719            0.995935         0.99561     0.998674                    0.996095            0.0198928       0.06114                    102.163   102.719            0.0610611\n",
       "4        0.04108                     0.995194           2.02244     2.02593            0.996337         0.995289    0.998053                    0.99588             0.0220851       0.0832251                  102.244   102.593            0.0830674\n",
       "5        0.05256                     0.99493            2.01573     2.0237             0.993031         0.995009    0.996956                    0.99569             0.0231406       0.106366                   101.573   102.37             0.10605\n",
       "6        0.1                         0.993286           2.01961     2.02176            0.994941         0.994155    0.996                       0.994962            0.0958103       0.202176                   101.961   102.176            0.201388\n",
       "7        0.15                        0.990269           2.02338     2.0223             0.9968           0.991949    0.996267                    0.993958            0.101169        0.303345                   102.338   102.23             0.302241\n",
       "8        0.2                         0.984066           2.01364     2.02014            0.992            0.987394    0.9952                      0.992317            0.100682        0.404027                   101.364   102.014            0.402135\n",
       "9        0.3                         0.95239            1.98522     2.0085             0.978            0.971705    0.989467                    0.985446            0.198522        0.60255                    98.5222   100.85             0.596321\n",
       "10       0.4                         0.867106           1.92351     1.98725            0.9476           0.914257    0.979                       0.967649            0.192351        0.794901                   92.3514   98.7252            0.778345\n",
       "11       0.5                         0.582235           1.61741     1.91328            0.7968           0.782506    0.94256                     0.93062             0.161741        0.956642                   61.7408   91.3284            0.900035\n",
       "12       0.6                         0.0667765          0.377558    1.65733            0.186            0.248018    0.816467                    0.816853            0.0377558       0.994398                   -62.2442  65.7329            0.777352\n",
       "13       0.7                         0.0125771          0.0186749   1.42324            0.0092           0.0304482   0.701143                    0.70451             0.00186749      0.996265                   -98.1325  42.3236            0.583935\n",
       "14       0.80048                     0.00574124         0.0185857   1.24692            0.00915605       0.00815736  0.614281                    0.6171              0.00186749      0.998133                   -98.1414  24.6917            0.389571\n",
       "15       0.9014                      0.00345923         0.0104592   1.10848            0.0051526        0.0045211   0.546084                    0.548516            0.00105554      0.999188                   -98.9541  10.8485            0.192739\n",
       "16       1                           0.000260908        0.00823481  1                  0.0040568        0.00264459  0.49264                     0.494693            0.000811952     1                          -99.1765  0                  0\n",
       "\n",
       "Scoring History: \n",
       "     timestamp            duration    number_of_trees    training_rmse        training_logloss     training_auc        training_pr_auc     training_lift       training_classification_error    training_custom    validation_rmse      validation_logloss    validation_auc      validation_pr_auc    validation_lift     validation_classification_error    validation_custom\n",
       "---  -------------------  ----------  -----------------  -------------------  -------------------  ------------------  ------------------  ------------------  -------------------------------  -----------------  -------------------  --------------------  ------------------  -------------------  ------------------  ---------------------------------  -------------------\n",
       "     2022-10-28 14:19:30  0.025 sec   0.0                0.5                  0.6931471805600504   0.5                 0.5022800000000001  1.0                 0.49772                          0.693147           0.5                  0.6931471805599932    0.5                 0.49263999999999997  1.0                 0.50736                            0.693147\n",
       "     2022-10-28 14:19:30  0.139 sec   1.0                0.39960097366259284  0.5084555759713797   0.9599604712562773  0.952882361784991   1.969944257252065   0.08508                          0.508456           0.4002938936840949   0.5095766191321826    0.9584754441256724  0.9501212341159861   2.007325610768287   0.08816                            0.509577\n",
       "     2022-10-28 14:19:30  0.179 sec   2.0                0.3370778235169717   0.40320483936633433  0.9693357563355386  0.9638432211755918  1.974978083921303   0.07869333333333334              0.403205           0.3378725612813438   0.4043147576521532    0.9689642384208382  0.9627452021209266   2.0180782041889165  0.08072                            0.404315\n",
       "     2022-10-28 14:19:30  0.228 sec   3.0                0.2957135469964664   0.33356511573213676  0.9735764390034864  0.9692971186564519  1.9823582956342736  0.07124                          0.333565           0.2972804020126436   0.33556896107331324   0.972758545365196   0.9672690232547145   2.0164812843739717  0.07328                            0.335569\n",
       "     2022-10-28 14:19:30  0.280 sec   4.0                0.2690565369849074   0.2867049786815229   0.9769385896258127  0.973666085141308   1.9824403701018989  0.06877333333333334              0.286705           0.2711929535273588   0.2894303318159203    0.9753679683707979  0.9709177118281687   2.0166126426753443  0.07028                            0.28943\n",
       "     2022-10-28 14:19:30  0.356 sec   5.0                0.2529770080480673   0.25489941558805895  0.9782348903583494  0.9753981493946517  1.9830012735164846  0.06936                          0.254899           0.2550436637594876   0.25752647933993555   0.976954942633842   0.9724197019498086   2.0204531445917966  0.06956                            0.257526\n",
       "     2022-10-28 14:19:30  0.434 sec   6.0                0.24305497030792145  0.23288588902305773  0.9792535639846858  0.9767332522440302  1.982694450495822   0.06864                          0.232886           0.24525713259217433  0.23572949157271123   0.9778084651737403  0.9737536794534183   2.0198804230789538  0.06872                            0.235729\n",
       "     2022-10-28 14:19:30  0.511 sec   7.0                0.23667027877522748  0.21730431422414767  0.9801004932398384  0.9779517651384936  1.9840124751203916  0.06553333333333333              0.217304           0.2388923863271564   0.22020638093402833   0.978794451615705   0.9751086853604877   2.018907507702571   0.06696                            0.220206\n",
       "     2022-10-28 14:19:30  0.587 sec   8.0                0.22742820294228833  0.2004641682608729   0.981689446457674   0.979584885842671   1.988342484694663   0.06172                          0.200464           0.23028641120487947  0.20405435411113004   0.9803330089878547  0.9772714003156485   2.021950613023709   0.06516                            0.204054\n",
       "     2022-10-28 14:19:31  0.676 sec   9.0                0.22350478838486162  0.1908877726426607   0.9823443926985863  0.9805219447464963  1.9883948484251914  0.0608                           0.190888           0.22675651465031937  0.19510476775550073   0.9808127993480623  0.9779093493480279   2.0222772849300505  0.06432                            0.195105\n",
       "---  ---                  ---         ---                ---                  ---                  ---                 ---                 ---                 ---                              ---                ---                  ---                   ---                 ---                  ---                 ---                                ---\n",
       "     2022-10-28 14:19:31  1.058 sec   13.0               0.20932986426153485  0.16188173031435207  0.9849208281386209  0.9835752931336927  1.9885342025018198  0.054933333333333334             0.161882           0.2140216611245386   0.16829754841446237   0.9829088919260484  0.9803702989731918   2.022757445741668   0.0584                             0.168298\n",
       "     2022-10-28 14:19:31  1.160 sec   14.0               0.20801159542673212  0.15872260659114573  0.9852451840387032  0.9839537573047635  1.9883086406824775  0.0544                           0.158723           0.2129618210780405   0.16569253851178234   0.9831326172024832  0.9806933766171955   2.0222772849300505  0.05852                            0.165693\n",
       "     2022-10-28 14:19:31  1.246 sec   15.0               0.20656261820821578  0.15561137441472306  0.9856594600307703  0.9846933313933812  1.9909213984231904  0.053853333333333336             0.155611           0.2118814635733488   0.16339145578701472   0.9833376856363834  0.9810162497973599   2.022782349187026   0.05792                            0.163391\n",
       "     2022-10-28 14:19:31  1.345 sec   16.0               0.2036095613587592   0.1517938621462097   0.9863473648236762  0.9853807508666774  1.9909213984231904  0.052026666666666666             0.151794           0.20991212994517833  0.1607776039482004    0.9837030416011293  0.9814827764043785   2.0217603117895417  0.0566                             0.160778\n",
       "     2022-10-28 14:19:31  1.462 sec   17.0               0.20012801070573083  0.1470822013139249   0.986886830974453   0.985910377892329   1.9909213984231904  0.05037333333333333              0.147082           0.2068740603495647   0.1565507907729844    0.9842040109503664  0.9821554851954529   2.029879831113998   0.05524                            0.156551\n",
       "     2022-10-28 14:19:31  1.566 sec   18.0               0.1952363704640891   0.14096879922102418  0.9877666946033187  0.9868323340675447  1.9909213984231904  0.04790666666666667              0.140969           0.20297695900076984  0.15166647071587894   0.9848695695625415  0.9829746186958338   2.029879831113998   0.05272                            0.151666\n",
       "     2022-10-28 14:19:32  1.684 sec   19.0               0.19436898343164097  0.1391751330056711   0.9880068361300588  0.9871283090191609  1.9909213984231904  0.04766666666666667              0.139175           0.2025051413752741   0.15058346705393647   0.9849703081904262  0.9829211071563434   2.029879831113998   0.05232                            0.150583\n",
       "     2022-10-28 14:19:32  1.802 sec   20.0               0.19194891175623852  0.13612128709347052  0.988400723075942   0.9875788628040455  1.9909213984231904  0.04669333333333334              0.136121           0.2009170028210971   0.14841515569437927   0.9852986065255842  0.9834889458246532   2.029879831113998   0.05108                            0.148415\n",
       "     2022-10-28 14:19:32  1.916 sec   21.0               0.18862571988282295  0.1319356174549523   0.9889610721498387  0.9882193402029281  1.9909213984231904  0.04445333333333333              0.131936           0.1983104566213673   0.1448947109368754    0.9857545901273815  0.9838307814948767   2.0217603117895417  0.0496                             0.144895\n",
       "     2022-10-28 14:19:32  2.057 sec   22.0               0.18643429859511995  0.12901406890397824  0.9893340459497513  0.988704085388505   1.9909213984231904  0.043813333333333336             0.129014           0.1964692560714765   0.14264211397729937   0.9860367984758349  0.9841777019906762   2.029879831113998   0.049                              0.142642\n",
       "[23 rows x 18 columns]\n",
       "\n",
       "\n",
       "Variable Importances: \n",
       "variable    relative_importance    scaled_importance    percentage\n",
       "----------  ---------------------  -------------------  ------------\n",
       "C11         70178.8                1                    0.426136\n",
       "C2          35732.7                0.509166             0.216974\n",
       "C13         21608.5                0.307906             0.13121\n",
       "C8          15654.5                0.223067             0.0950568\n",
       "C3          12402.3                0.176724             0.0753084\n",
       "C19         5739.76                0.0817878            0.0348527\n",
       "C5          2835.92                0.0404099            0.0172201\n",
       "C6          88.9352                0.00126727           0.000540028\n",
       "C12         51.9876                0.000740789          0.000315677\n",
       "C17         49.1711                0.000700655          0.000298574\n",
       "C7          48.0521                0.000684711          0.00029178\n",
       "C14         46.5472                0.000663267          0.000282642\n",
       "C10         43.8789                0.000625244          0.000266439\n",
       "C4          42.4347                0.000604665          0.00025767\n",
       "C18         39.3953                0.000561357          0.000239214\n",
       "C1          32.554                 0.000463873          0.000197673\n",
       "C15         25.9528                0.00036981           0.000157589\n",
       "C20         23.1192                0.000329434          0.000140383\n",
       "C16         22.9984                0.000327711          0.00013965\n",
       "C9          18.9113                0.000269474          0.000114832\n",
       "\n",
       "[tips]\n",
       "Use `model.explain()` to inspect the model.\n",
       "--\n",
       "Use `h2o.display.toggle_user_tips()` to switch on/off this section."
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_eval = H2OXGBoostEstimator(ntrees=1000, max_depth=6, score_each_iteration=True, \n",
    "                                 eval_metric=\"logloss\",\n",
    "                                 stopping_rounds=3, stopping_tolerance=0.05, stopping_metric=\"custom\")\n",
    "model_eval.train(y=\"y\", training_frame=train, validation_frame=valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8582fbb0",
   "metadata": {},
   "source": [
    "The scoring history for model with `eval_metric=\"logloss\"`  will look similar to the scoring history of the first model. This is expected - we didn't actually changed the training behavior, we only changed the source of the values that trigger early stopping. In this case, we are stopping on values of `validation_custom`. This value correspond to the value calculated and returned by XGBoost. It should be close to H2O's own `validation_logloss` value, there can be only a small difference caused by a different precision in XGBoost and H2O (the values should be within absolute tolerance of 1e-5). This is, however, something to keep in mind. There can be edge cases where H2O metric will differ slightly from the conceptually same XGBoost metric and this might cause the models to stop at a different iteration.\n",
    "\n",
    "The scoring history will also have value of `eval_metric` for the training frame - see column `training_custom`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "46135033",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>duration</th>\n",
       "      <th>number_of_trees</th>\n",
       "      <th>training_rmse</th>\n",
       "      <th>training_logloss</th>\n",
       "      <th>training_auc</th>\n",
       "      <th>training_pr_auc</th>\n",
       "      <th>training_lift</th>\n",
       "      <th>training_classification_error</th>\n",
       "      <th>training_custom</th>\n",
       "      <th>validation_rmse</th>\n",
       "      <th>validation_logloss</th>\n",
       "      <th>validation_auc</th>\n",
       "      <th>validation_pr_auc</th>\n",
       "      <th>validation_lift</th>\n",
       "      <th>validation_classification_error</th>\n",
       "      <th>validation_custom</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.025 sec</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.693147</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.502280</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.497720</td>\n",
       "      <td>0.693147</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.693147</td>\n",
       "      <td>0.500000</td>\n",
       "      <td>0.492640</td>\n",
       "      <td>1.000000</td>\n",
       "      <td>0.50736</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.139 sec</td>\n",
       "      <td>1.0</td>\n",
       "      <td>0.399601</td>\n",
       "      <td>0.508456</td>\n",
       "      <td>0.959960</td>\n",
       "      <td>0.952882</td>\n",
       "      <td>1.969944</td>\n",
       "      <td>0.085080</td>\n",
       "      <td>0.508456</td>\n",
       "      <td>0.400294</td>\n",
       "      <td>0.509577</td>\n",
       "      <td>0.958475</td>\n",
       "      <td>0.950121</td>\n",
       "      <td>2.007326</td>\n",
       "      <td>0.08816</td>\n",
       "      <td>0.509577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.179 sec</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.337078</td>\n",
       "      <td>0.403205</td>\n",
       "      <td>0.969336</td>\n",
       "      <td>0.963843</td>\n",
       "      <td>1.974978</td>\n",
       "      <td>0.078693</td>\n",
       "      <td>0.403205</td>\n",
       "      <td>0.337873</td>\n",
       "      <td>0.404315</td>\n",
       "      <td>0.968964</td>\n",
       "      <td>0.962745</td>\n",
       "      <td>2.018078</td>\n",
       "      <td>0.08072</td>\n",
       "      <td>0.404315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.228 sec</td>\n",
       "      <td>3.0</td>\n",
       "      <td>0.295714</td>\n",
       "      <td>0.333565</td>\n",
       "      <td>0.973576</td>\n",
       "      <td>0.969297</td>\n",
       "      <td>1.982358</td>\n",
       "      <td>0.071240</td>\n",
       "      <td>0.333565</td>\n",
       "      <td>0.297280</td>\n",
       "      <td>0.335569</td>\n",
       "      <td>0.972759</td>\n",
       "      <td>0.967269</td>\n",
       "      <td>2.016481</td>\n",
       "      <td>0.07328</td>\n",
       "      <td>0.335569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.280 sec</td>\n",
       "      <td>4.0</td>\n",
       "      <td>0.269057</td>\n",
       "      <td>0.286705</td>\n",
       "      <td>0.976939</td>\n",
       "      <td>0.973666</td>\n",
       "      <td>1.982440</td>\n",
       "      <td>0.068773</td>\n",
       "      <td>0.286705</td>\n",
       "      <td>0.271193</td>\n",
       "      <td>0.289430</td>\n",
       "      <td>0.975368</td>\n",
       "      <td>0.970918</td>\n",
       "      <td>2.016613</td>\n",
       "      <td>0.07028</td>\n",
       "      <td>0.289430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.356 sec</td>\n",
       "      <td>5.0</td>\n",
       "      <td>0.252977</td>\n",
       "      <td>0.254899</td>\n",
       "      <td>0.978235</td>\n",
       "      <td>0.975398</td>\n",
       "      <td>1.983001</td>\n",
       "      <td>0.069360</td>\n",
       "      <td>0.254899</td>\n",
       "      <td>0.255044</td>\n",
       "      <td>0.257526</td>\n",
       "      <td>0.976955</td>\n",
       "      <td>0.972420</td>\n",
       "      <td>2.020453</td>\n",
       "      <td>0.06956</td>\n",
       "      <td>0.257526</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.434 sec</td>\n",
       "      <td>6.0</td>\n",
       "      <td>0.243055</td>\n",
       "      <td>0.232886</td>\n",
       "      <td>0.979254</td>\n",
       "      <td>0.976733</td>\n",
       "      <td>1.982694</td>\n",
       "      <td>0.068640</td>\n",
       "      <td>0.232886</td>\n",
       "      <td>0.245257</td>\n",
       "      <td>0.235729</td>\n",
       "      <td>0.977808</td>\n",
       "      <td>0.973754</td>\n",
       "      <td>2.019880</td>\n",
       "      <td>0.06872</td>\n",
       "      <td>0.235729</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.511 sec</td>\n",
       "      <td>7.0</td>\n",
       "      <td>0.236670</td>\n",
       "      <td>0.217304</td>\n",
       "      <td>0.980100</td>\n",
       "      <td>0.977952</td>\n",
       "      <td>1.984012</td>\n",
       "      <td>0.065533</td>\n",
       "      <td>0.217304</td>\n",
       "      <td>0.238892</td>\n",
       "      <td>0.220206</td>\n",
       "      <td>0.978794</td>\n",
       "      <td>0.975109</td>\n",
       "      <td>2.018908</td>\n",
       "      <td>0.06696</td>\n",
       "      <td>0.220206</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:30</td>\n",
       "      <td>0.587 sec</td>\n",
       "      <td>8.0</td>\n",
       "      <td>0.227428</td>\n",
       "      <td>0.200464</td>\n",
       "      <td>0.981689</td>\n",
       "      <td>0.979585</td>\n",
       "      <td>1.988342</td>\n",
       "      <td>0.061720</td>\n",
       "      <td>0.200464</td>\n",
       "      <td>0.230286</td>\n",
       "      <td>0.204054</td>\n",
       "      <td>0.980333</td>\n",
       "      <td>0.977271</td>\n",
       "      <td>2.021951</td>\n",
       "      <td>0.06516</td>\n",
       "      <td>0.204054</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>0.676 sec</td>\n",
       "      <td>9.0</td>\n",
       "      <td>0.223505</td>\n",
       "      <td>0.190888</td>\n",
       "      <td>0.982344</td>\n",
       "      <td>0.980522</td>\n",
       "      <td>1.988395</td>\n",
       "      <td>0.060800</td>\n",
       "      <td>0.190888</td>\n",
       "      <td>0.226757</td>\n",
       "      <td>0.195105</td>\n",
       "      <td>0.980813</td>\n",
       "      <td>0.977909</td>\n",
       "      <td>2.022277</td>\n",
       "      <td>0.06432</td>\n",
       "      <td>0.195105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>0.773 sec</td>\n",
       "      <td>10.0</td>\n",
       "      <td>0.220040</td>\n",
       "      <td>0.182967</td>\n",
       "      <td>0.983000</td>\n",
       "      <td>0.981210</td>\n",
       "      <td>1.988298</td>\n",
       "      <td>0.059227</td>\n",
       "      <td>0.182967</td>\n",
       "      <td>0.223656</td>\n",
       "      <td>0.187705</td>\n",
       "      <td>0.981356</td>\n",
       "      <td>0.978627</td>\n",
       "      <td>2.021888</td>\n",
       "      <td>0.06240</td>\n",
       "      <td>0.187705</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>0.854 sec</td>\n",
       "      <td>11.0</td>\n",
       "      <td>0.214926</td>\n",
       "      <td>0.173463</td>\n",
       "      <td>0.983878</td>\n",
       "      <td>0.982224</td>\n",
       "      <td>1.988819</td>\n",
       "      <td>0.057027</td>\n",
       "      <td>0.173463</td>\n",
       "      <td>0.218899</td>\n",
       "      <td>0.178748</td>\n",
       "      <td>0.982148</td>\n",
       "      <td>0.979434</td>\n",
       "      <td>2.023615</td>\n",
       "      <td>0.06084</td>\n",
       "      <td>0.178748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>0.943 sec</td>\n",
       "      <td>12.0</td>\n",
       "      <td>0.212074</td>\n",
       "      <td>0.167261</td>\n",
       "      <td>0.984348</td>\n",
       "      <td>0.982791</td>\n",
       "      <td>1.988617</td>\n",
       "      <td>0.055987</td>\n",
       "      <td>0.167261</td>\n",
       "      <td>0.216237</td>\n",
       "      <td>0.172957</td>\n",
       "      <td>0.982491</td>\n",
       "      <td>0.979711</td>\n",
       "      <td>2.022975</td>\n",
       "      <td>0.05932</td>\n",
       "      <td>0.172957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>1.058 sec</td>\n",
       "      <td>13.0</td>\n",
       "      <td>0.209330</td>\n",
       "      <td>0.161882</td>\n",
       "      <td>0.984921</td>\n",
       "      <td>0.983575</td>\n",
       "      <td>1.988534</td>\n",
       "      <td>0.054933</td>\n",
       "      <td>0.161882</td>\n",
       "      <td>0.214022</td>\n",
       "      <td>0.168298</td>\n",
       "      <td>0.982909</td>\n",
       "      <td>0.980370</td>\n",
       "      <td>2.022757</td>\n",
       "      <td>0.05840</td>\n",
       "      <td>0.168298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>1.160 sec</td>\n",
       "      <td>14.0</td>\n",
       "      <td>0.208012</td>\n",
       "      <td>0.158723</td>\n",
       "      <td>0.985245</td>\n",
       "      <td>0.983954</td>\n",
       "      <td>1.988309</td>\n",
       "      <td>0.054400</td>\n",
       "      <td>0.158723</td>\n",
       "      <td>0.212962</td>\n",
       "      <td>0.165693</td>\n",
       "      <td>0.983133</td>\n",
       "      <td>0.980693</td>\n",
       "      <td>2.022277</td>\n",
       "      <td>0.05852</td>\n",
       "      <td>0.165693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>1.246 sec</td>\n",
       "      <td>15.0</td>\n",
       "      <td>0.206563</td>\n",
       "      <td>0.155611</td>\n",
       "      <td>0.985659</td>\n",
       "      <td>0.984693</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.053853</td>\n",
       "      <td>0.155611</td>\n",
       "      <td>0.211881</td>\n",
       "      <td>0.163391</td>\n",
       "      <td>0.983338</td>\n",
       "      <td>0.981016</td>\n",
       "      <td>2.022782</td>\n",
       "      <td>0.05792</td>\n",
       "      <td>0.163391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>1.345 sec</td>\n",
       "      <td>16.0</td>\n",
       "      <td>0.203610</td>\n",
       "      <td>0.151794</td>\n",
       "      <td>0.986347</td>\n",
       "      <td>0.985381</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.052027</td>\n",
       "      <td>0.151794</td>\n",
       "      <td>0.209912</td>\n",
       "      <td>0.160778</td>\n",
       "      <td>0.983703</td>\n",
       "      <td>0.981483</td>\n",
       "      <td>2.021760</td>\n",
       "      <td>0.05660</td>\n",
       "      <td>0.160778</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>1.462 sec</td>\n",
       "      <td>17.0</td>\n",
       "      <td>0.200128</td>\n",
       "      <td>0.147082</td>\n",
       "      <td>0.986887</td>\n",
       "      <td>0.985910</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.050373</td>\n",
       "      <td>0.147082</td>\n",
       "      <td>0.206874</td>\n",
       "      <td>0.156551</td>\n",
       "      <td>0.984204</td>\n",
       "      <td>0.982155</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.05524</td>\n",
       "      <td>0.156551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:31</td>\n",
       "      <td>1.566 sec</td>\n",
       "      <td>18.0</td>\n",
       "      <td>0.195236</td>\n",
       "      <td>0.140969</td>\n",
       "      <td>0.987767</td>\n",
       "      <td>0.986832</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.047907</td>\n",
       "      <td>0.140969</td>\n",
       "      <td>0.202977</td>\n",
       "      <td>0.151666</td>\n",
       "      <td>0.984870</td>\n",
       "      <td>0.982975</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.05272</td>\n",
       "      <td>0.151666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:32</td>\n",
       "      <td>1.684 sec</td>\n",
       "      <td>19.0</td>\n",
       "      <td>0.194369</td>\n",
       "      <td>0.139175</td>\n",
       "      <td>0.988007</td>\n",
       "      <td>0.987128</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.047667</td>\n",
       "      <td>0.139175</td>\n",
       "      <td>0.202505</td>\n",
       "      <td>0.150583</td>\n",
       "      <td>0.984970</td>\n",
       "      <td>0.982921</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.05232</td>\n",
       "      <td>0.150583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:32</td>\n",
       "      <td>1.802 sec</td>\n",
       "      <td>20.0</td>\n",
       "      <td>0.191949</td>\n",
       "      <td>0.136121</td>\n",
       "      <td>0.988401</td>\n",
       "      <td>0.987579</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.046693</td>\n",
       "      <td>0.136121</td>\n",
       "      <td>0.200917</td>\n",
       "      <td>0.148415</td>\n",
       "      <td>0.985299</td>\n",
       "      <td>0.983489</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.05108</td>\n",
       "      <td>0.148415</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:32</td>\n",
       "      <td>1.916 sec</td>\n",
       "      <td>21.0</td>\n",
       "      <td>0.188626</td>\n",
       "      <td>0.131936</td>\n",
       "      <td>0.988961</td>\n",
       "      <td>0.988219</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.044453</td>\n",
       "      <td>0.131936</td>\n",
       "      <td>0.198310</td>\n",
       "      <td>0.144895</td>\n",
       "      <td>0.985755</td>\n",
       "      <td>0.983831</td>\n",
       "      <td>2.021760</td>\n",
       "      <td>0.04960</td>\n",
       "      <td>0.144895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:32</td>\n",
       "      <td>2.057 sec</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.186434</td>\n",
       "      <td>0.129014</td>\n",
       "      <td>0.989334</td>\n",
       "      <td>0.988704</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.043813</td>\n",
       "      <td>0.129014</td>\n",
       "      <td>0.196469</td>\n",
       "      <td>0.142642</td>\n",
       "      <td>0.986037</td>\n",
       "      <td>0.984178</td>\n",
       "      <td>2.029880</td>\n",
       "      <td>0.04900</td>\n",
       "      <td>0.142642</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                timestamp    duration  number_of_trees  training_rmse  \\\n",
       "0     2022-10-28 14:19:30   0.025 sec              0.0       0.500000   \n",
       "1     2022-10-28 14:19:30   0.139 sec              1.0       0.399601   \n",
       "2     2022-10-28 14:19:30   0.179 sec              2.0       0.337078   \n",
       "3     2022-10-28 14:19:30   0.228 sec              3.0       0.295714   \n",
       "4     2022-10-28 14:19:30   0.280 sec              4.0       0.269057   \n",
       "5     2022-10-28 14:19:30   0.356 sec              5.0       0.252977   \n",
       "6     2022-10-28 14:19:30   0.434 sec              6.0       0.243055   \n",
       "7     2022-10-28 14:19:30   0.511 sec              7.0       0.236670   \n",
       "8     2022-10-28 14:19:30   0.587 sec              8.0       0.227428   \n",
       "9     2022-10-28 14:19:31   0.676 sec              9.0       0.223505   \n",
       "10    2022-10-28 14:19:31   0.773 sec             10.0       0.220040   \n",
       "11    2022-10-28 14:19:31   0.854 sec             11.0       0.214926   \n",
       "12    2022-10-28 14:19:31   0.943 sec             12.0       0.212074   \n",
       "13    2022-10-28 14:19:31   1.058 sec             13.0       0.209330   \n",
       "14    2022-10-28 14:19:31   1.160 sec             14.0       0.208012   \n",
       "15    2022-10-28 14:19:31   1.246 sec             15.0       0.206563   \n",
       "16    2022-10-28 14:19:31   1.345 sec             16.0       0.203610   \n",
       "17    2022-10-28 14:19:31   1.462 sec             17.0       0.200128   \n",
       "18    2022-10-28 14:19:31   1.566 sec             18.0       0.195236   \n",
       "19    2022-10-28 14:19:32   1.684 sec             19.0       0.194369   \n",
       "20    2022-10-28 14:19:32   1.802 sec             20.0       0.191949   \n",
       "21    2022-10-28 14:19:32   1.916 sec             21.0       0.188626   \n",
       "22    2022-10-28 14:19:32   2.057 sec             22.0       0.186434   \n",
       "\n",
       "    training_logloss  training_auc  training_pr_auc  training_lift  \\\n",
       "0           0.693147      0.500000         0.502280       1.000000   \n",
       "1           0.508456      0.959960         0.952882       1.969944   \n",
       "2           0.403205      0.969336         0.963843       1.974978   \n",
       "3           0.333565      0.973576         0.969297       1.982358   \n",
       "4           0.286705      0.976939         0.973666       1.982440   \n",
       "5           0.254899      0.978235         0.975398       1.983001   \n",
       "6           0.232886      0.979254         0.976733       1.982694   \n",
       "7           0.217304      0.980100         0.977952       1.984012   \n",
       "8           0.200464      0.981689         0.979585       1.988342   \n",
       "9           0.190888      0.982344         0.980522       1.988395   \n",
       "10          0.182967      0.983000         0.981210       1.988298   \n",
       "11          0.173463      0.983878         0.982224       1.988819   \n",
       "12          0.167261      0.984348         0.982791       1.988617   \n",
       "13          0.161882      0.984921         0.983575       1.988534   \n",
       "14          0.158723      0.985245         0.983954       1.988309   \n",
       "15          0.155611      0.985659         0.984693       1.990921   \n",
       "16          0.151794      0.986347         0.985381       1.990921   \n",
       "17          0.147082      0.986887         0.985910       1.990921   \n",
       "18          0.140969      0.987767         0.986832       1.990921   \n",
       "19          0.139175      0.988007         0.987128       1.990921   \n",
       "20          0.136121      0.988401         0.987579       1.990921   \n",
       "21          0.131936      0.988961         0.988219       1.990921   \n",
       "22          0.129014      0.989334         0.988704       1.990921   \n",
       "\n",
       "    training_classification_error  training_custom  validation_rmse  \\\n",
       "0                        0.497720         0.693147         0.500000   \n",
       "1                        0.085080         0.508456         0.400294   \n",
       "2                        0.078693         0.403205         0.337873   \n",
       "3                        0.071240         0.333565         0.297280   \n",
       "4                        0.068773         0.286705         0.271193   \n",
       "5                        0.069360         0.254899         0.255044   \n",
       "6                        0.068640         0.232886         0.245257   \n",
       "7                        0.065533         0.217304         0.238892   \n",
       "8                        0.061720         0.200464         0.230286   \n",
       "9                        0.060800         0.190888         0.226757   \n",
       "10                       0.059227         0.182967         0.223656   \n",
       "11                       0.057027         0.173463         0.218899   \n",
       "12                       0.055987         0.167261         0.216237   \n",
       "13                       0.054933         0.161882         0.214022   \n",
       "14                       0.054400         0.158723         0.212962   \n",
       "15                       0.053853         0.155611         0.211881   \n",
       "16                       0.052027         0.151794         0.209912   \n",
       "17                       0.050373         0.147082         0.206874   \n",
       "18                       0.047907         0.140969         0.202977   \n",
       "19                       0.047667         0.139175         0.202505   \n",
       "20                       0.046693         0.136121         0.200917   \n",
       "21                       0.044453         0.131936         0.198310   \n",
       "22                       0.043813         0.129014         0.196469   \n",
       "\n",
       "    validation_logloss  validation_auc  validation_pr_auc  validation_lift  \\\n",
       "0             0.693147        0.500000           0.492640         1.000000   \n",
       "1             0.509577        0.958475           0.950121         2.007326   \n",
       "2             0.404315        0.968964           0.962745         2.018078   \n",
       "3             0.335569        0.972759           0.967269         2.016481   \n",
       "4             0.289430        0.975368           0.970918         2.016613   \n",
       "5             0.257526        0.976955           0.972420         2.020453   \n",
       "6             0.235729        0.977808           0.973754         2.019880   \n",
       "7             0.220206        0.978794           0.975109         2.018908   \n",
       "8             0.204054        0.980333           0.977271         2.021951   \n",
       "9             0.195105        0.980813           0.977909         2.022277   \n",
       "10            0.187705        0.981356           0.978627         2.021888   \n",
       "11            0.178748        0.982148           0.979434         2.023615   \n",
       "12            0.172957        0.982491           0.979711         2.022975   \n",
       "13            0.168298        0.982909           0.980370         2.022757   \n",
       "14            0.165693        0.983133           0.980693         2.022277   \n",
       "15            0.163391        0.983338           0.981016         2.022782   \n",
       "16            0.160778        0.983703           0.981483         2.021760   \n",
       "17            0.156551        0.984204           0.982155         2.029880   \n",
       "18            0.151666        0.984870           0.982975         2.029880   \n",
       "19            0.150583        0.984970           0.982921         2.029880   \n",
       "20            0.148415        0.985299           0.983489         2.029880   \n",
       "21            0.144895        0.985755           0.983831         2.021760   \n",
       "22            0.142642        0.986037           0.984178         2.029880   \n",
       "\n",
       "    validation_classification_error  validation_custom  \n",
       "0                           0.50736           0.693147  \n",
       "1                           0.08816           0.509577  \n",
       "2                           0.08072           0.404315  \n",
       "3                           0.07328           0.335569  \n",
       "4                           0.07028           0.289430  \n",
       "5                           0.06956           0.257526  \n",
       "6                           0.06872           0.235729  \n",
       "7                           0.06696           0.220206  \n",
       "8                           0.06516           0.204054  \n",
       "9                           0.06432           0.195105  \n",
       "10                          0.06240           0.187705  \n",
       "11                          0.06084           0.178748  \n",
       "12                          0.05932           0.172957  \n",
       "13                          0.05840           0.168298  \n",
       "14                          0.05852           0.165693  \n",
       "15                          0.05792           0.163391  \n",
       "16                          0.05660           0.160778  \n",
       "17                          0.05524           0.156551  \n",
       "18                          0.05272           0.151666  \n",
       "19                          0.05232           0.150583  \n",
       "20                          0.05108           0.148415  \n",
       "21                          0.04960           0.144895  \n",
       "22                          0.04900           0.142642  "
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_eval.scoring_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "78f7a080",
   "metadata": {},
   "source": [
    "##### Train XGBoost model, use eval_metric=\"logloss\" for early stopping and disable H2O metrics to speed-up model training\n",
    "\n",
    "In this example we will keep the same parameters as in the previous case and add flag `score_eval_metric_only=True`. This flag will instruct H2O to disable its own scoring and solely rely on `eval_metric` for early stopping and recording the scoring history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "2966b1f4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "xgboost Model Build progress: |██████████████████████████████████████████████████| (done) 100%\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<pre style='margin: 1em 0 1em 0;'>Model Details\n",
       "=============\n",
       "H2OXGBoostEstimator : XGBoost\n",
       "Model Key: XGBoost_model_python_1666962310297_616\n",
       "</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-20.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-20 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-20 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-20 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-20 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-20 .h2o-table th,\n",
       "#h2o-table-20 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-20 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-20\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Model Summary: </caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>number_of_trees</th></tr></thead>\n",
       "    <tbody><tr><td></td>\n",
       "<td>22.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'><pre style='margin: 1em 0 1em 0;'>ModelMetricsBinomial: xgboost\n",
       "** Reported on train data. **\n",
       "\n",
       "MSE: 0.03475774769265435\n",
       "RMSE: 0.18643429859511995\n",
       "LogLoss: 0.12901406890397824\n",
       "Mean Per-Class Error: 0.04387881533186782\n",
       "AUC: 0.9893340459497513\n",
       "AUCPR: 0.988704085388505\n",
       "Gini: 0.9786680918995025</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-21.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-21 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-21 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-21 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-21 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-21 .h2o-table th,\n",
       "#h2o-table-21 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-21 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-21\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.5101676438305829</caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>0</th>\n",
       "<th>1</th>\n",
       "<th>Error</th>\n",
       "<th>Rate</th></tr></thead>\n",
       "    <tbody><tr><td>0</td>\n",
       "<td>35155.0</td>\n",
       "<td>2174.0</td>\n",
       "<td>0.0582</td>\n",
       "<td> (2174.0/37329.0)</td></tr>\n",
       "<tr><td>1</td>\n",
       "<td>1112.0</td>\n",
       "<td>36559.0</td>\n",
       "<td>0.0295</td>\n",
       "<td> (1112.0/37671.0)</td></tr>\n",
       "<tr><td>Total</td>\n",
       "<td>36267.0</td>\n",
       "<td>38733.0</td>\n",
       "<td>0.0438</td>\n",
       "<td> (3286.0/75000.0)</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-22.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-22 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-22 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-22 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-22 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-22 .h2o-table th,\n",
       "#h2o-table-22 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-22 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-22\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Maximum Metrics: Maximum metrics at their respective thresholds</caption>\n",
       "    <thead><tr><th>metric</th>\n",
       "<th>threshold</th>\n",
       "<th>value</th>\n",
       "<th>idx</th></tr></thead>\n",
       "    <tbody><tr><td>max f1</td>\n",
       "<td>0.5101676</td>\n",
       "<td>0.9569918</td>\n",
       "<td>214.0</td></tr>\n",
       "<tr><td>max f2</td>\n",
       "<td>0.3082806</td>\n",
       "<td>0.9715081</td>\n",
       "<td>263.0</td></tr>\n",
       "<tr><td>max f0point5</td>\n",
       "<td>0.7532607</td>\n",
       "<td>0.9592084</td>\n",
       "<td>147.0</td></tr>\n",
       "<tr><td>max accuracy</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9564</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max precision</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max recall</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max specificity</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max absolute_mcc</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9129277</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max min_per_class_accuracy</td>\n",
       "<td>0.6239853</td>\n",
       "<td>0.9548073</td>\n",
       "<td>184.0</td></tr>\n",
       "<tr><td>max mean_per_class_accuracy</td>\n",
       "<td>0.5561712</td>\n",
       "<td>0.9563597</td>\n",
       "<td>201.0</td></tr>\n",
       "<tr><td>max tns</td>\n",
       "<td>0.9974565</td>\n",
       "<td>37329.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fns</td>\n",
       "<td>0.9974565</td>\n",
       "<td>37460.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fps</td>\n",
       "<td>0.0016028</td>\n",
       "<td>37329.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tps</td>\n",
       "<td>0.0016028</td>\n",
       "<td>37671.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tnr</td>\n",
       "<td>0.9974565</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fnr</td>\n",
       "<td>0.9974565</td>\n",
       "<td>0.9943989</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fpr</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tpr</td>\n",
       "<td>0.0016028</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-23.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-23 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-23 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-23 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-23 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-23 .h2o-table th,\n",
       "#h2o-table-23 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-23 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-23\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Gains/Lift Table: Avg response rate: 50.23 %, avg score: 50.22 %</caption>\n",
       "    <thead><tr><th>group</th>\n",
       "<th>cumulative_data_fraction</th>\n",
       "<th>lower_threshold</th>\n",
       "<th>lift</th>\n",
       "<th>cumulative_lift</th>\n",
       "<th>response_rate</th>\n",
       "<th>score</th>\n",
       "<th>cumulative_response_rate</th>\n",
       "<th>cumulative_score</th>\n",
       "<th>capture_rate</th>\n",
       "<th>cumulative_capture_rate</th>\n",
       "<th>gain</th>\n",
       "<th>cumulative_gain</th>\n",
       "<th>kolmogorov_smirnov</th></tr></thead>\n",
       "    <tbody><tr><td>1</td>\n",
       "<td>0.0101733</td>\n",
       "<td>0.9961097</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967516</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967516</td>\n",
       "<td>0.0202543</td>\n",
       "<td>0.0202543</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0202543</td></tr>\n",
       "<tr><td>2</td>\n",
       "<td>0.0200267</td>\n",
       "<td>0.9957153</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9958894</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9963274</td>\n",
       "<td>0.0196172</td>\n",
       "<td>0.0398715</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0398715</td></tr>\n",
       "<tr><td>3</td>\n",
       "<td>0.03068</td>\n",
       "<td>0.9953964</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.9909214</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9955495</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9960573</td>\n",
       "<td>0.0212099</td>\n",
       "<td>0.0610815</td>\n",
       "<td>99.0921398</td>\n",
       "<td>99.0921398</td>\n",
       "<td>0.0610815</td></tr>\n",
       "<tr><td>4</td>\n",
       "<td>0.0401067</td>\n",
       "<td>0.9951944</td>\n",
       "<td>1.9881054</td>\n",
       "<td>1.9902595</td>\n",
       "<td>0.9985856</td>\n",
       "<td>0.9952708</td>\n",
       "<td>0.9996676</td>\n",
       "<td>0.9958724</td>\n",
       "<td>0.0187412</td>\n",
       "<td>0.0798227</td>\n",
       "<td>98.8105385</td>\n",
       "<td>99.0259523</td>\n",
       "<td>0.0797959</td></tr>\n",
       "<tr><td>5</td>\n",
       "<td>0.0517733</td>\n",
       "<td>0.9949297</td>\n",
       "<td>1.9863707</td>\n",
       "<td>1.9893832</td>\n",
       "<td>0.9977143</td>\n",
       "<td>0.9950084</td>\n",
       "<td>0.9992274</td>\n",
       "<td>0.9956777</td>\n",
       "<td>0.0231743</td>\n",
       "<td>0.1029970</td>\n",
       "<td>98.6370721</td>\n",
       "<td>98.9383216</td>\n",
       "<td>0.1029166</td></tr>\n",
       "<tr><td>6</td>\n",
       "<td>0.1000133</td>\n",
       "<td>0.9933097</td>\n",
       "<td>1.9881700</td>\n",
       "<td>1.9887980</td>\n",
       "<td>0.9986180</td>\n",
       "<td>0.9941657</td>\n",
       "<td>0.9989335</td>\n",
       "<td>0.9949484</td>\n",
       "<td>0.0959093</td>\n",
       "<td>0.1989063</td>\n",
       "<td>98.8169987</td>\n",
       "<td>98.8798032</td>\n",
       "<td>0.1986920</td></tr>\n",
       "<tr><td>7</td>\n",
       "<td>0.15008</td>\n",
       "<td>0.9904984</td>\n",
       "<td>1.9840287</td>\n",
       "<td>1.9872070</td>\n",
       "<td>0.9965379</td>\n",
       "<td>0.9920577</td>\n",
       "<td>0.9981343</td>\n",
       "<td>0.9939841</td>\n",
       "<td>0.0993337</td>\n",
       "<td>0.2982400</td>\n",
       "<td>98.4028728</td>\n",
       "<td>98.7206993</td>\n",
       "<td>0.2976775</td></tr>\n",
       "<tr><td>8</td>\n",
       "<td>0.2000133</td>\n",
       "<td>0.9849605</td>\n",
       "<td>1.9824155</td>\n",
       "<td>1.9860108</td>\n",
       "<td>0.9957276</td>\n",
       "<td>0.9879125</td>\n",
       "<td>0.9975335</td>\n",
       "<td>0.9924683</td>\n",
       "<td>0.0989886</td>\n",
       "<td>0.3972286</td>\n",
       "<td>98.2415459</td>\n",
       "<td>98.6010786</td>\n",
       "<td>0.3962375</td></tr>\n",
       "<tr><td>9</td>\n",
       "<td>0.30004</td>\n",
       "<td>0.9561046</td>\n",
       "<td>1.9673021</td>\n",
       "<td>1.9797737</td>\n",
       "<td>0.9881365</td>\n",
       "<td>0.9740387</td>\n",
       "<td>0.9944007</td>\n",
       "<td>0.9863243</td>\n",
       "<td>0.1967827</td>\n",
       "<td>0.5940113</td>\n",
       "<td>96.7302096</td>\n",
       "<td>97.9773725</td>\n",
       "<td>0.5906359</td></tr>\n",
       "<tr><td>10</td>\n",
       "<td>0.4000133</td>\n",
       "<td>0.8770613</td>\n",
       "<td>1.9123254</td>\n",
       "<td>1.9629167</td>\n",
       "<td>0.9605228</td>\n",
       "<td>0.9210076</td>\n",
       "<td>0.9859338</td>\n",
       "<td>0.9700000</td>\n",
       "<td>0.1911815</td>\n",
       "<td>0.7851929</td>\n",
       "<td>91.2325408</td>\n",
       "<td>96.2916704</td>\n",
       "<td>0.7738880</td></tr>\n",
       "<tr><td>11</td>\n",
       "<td>0.5000133</td>\n",
       "<td>0.6383476</td>\n",
       "<td>1.6763558</td>\n",
       "<td>1.9056061</td>\n",
       "<td>0.842</td>\n",
       "<td>0.8051186</td>\n",
       "<td>0.9571478</td>\n",
       "<td>0.9370246</td>\n",
       "<td>0.1676356</td>\n",
       "<td>0.9528284</td>\n",
       "<td>67.6355817</td>\n",
       "<td>90.5606055</td>\n",
       "<td>0.9097788</td></tr>\n",
       "<tr><td>12</td>\n",
       "<td>0.6</td>\n",
       "<td>0.0781245</td>\n",
       "<td>0.4261140</td>\n",
       "<td>1.6590569</td>\n",
       "<td>0.2140285</td>\n",
       "<td>0.2876185</td>\n",
       "<td>0.8333111</td>\n",
       "<td>0.8288047</td>\n",
       "<td>0.0426057</td>\n",
       "<td>0.9954342</td>\n",
       "<td>-57.3886006</td>\n",
       "<td>65.9056923</td>\n",
       "<td>0.7944912</td></tr>\n",
       "<tr><td>13</td>\n",
       "<td>0.7000133</td>\n",
       "<td>0.0128222</td>\n",
       "<td>0.0238879</td>\n",
       "<td>1.4254346</td>\n",
       "<td>0.0119984</td>\n",
       "<td>0.0339001</td>\n",
       "<td>0.7159673</td>\n",
       "<td>0.7152339</td>\n",
       "<td>0.0023891</td>\n",
       "<td>0.9978233</td>\n",
       "<td>-97.6112128</td>\n",
       "<td>42.5434648</td>\n",
       "<td>0.5983483</td></tr>\n",
       "<tr><td>14</td>\n",
       "<td>0.80008</td>\n",
       "<td>0.0057930</td>\n",
       "<td>0.0122028</td>\n",
       "<td>1.2486806</td>\n",
       "<td>0.0061292</td>\n",
       "<td>0.0084165</td>\n",
       "<td>0.6271873</td>\n",
       "<td>0.6268317</td>\n",
       "<td>0.0012211</td>\n",
       "<td>0.9990444</td>\n",
       "<td>-98.7797151</td>\n",
       "<td>24.8680579</td>\n",
       "<td>0.3997516</td></tr>\n",
       "<tr><td>15</td>\n",
       "<td>0.9</td>\n",
       "<td>0.0034428</td>\n",
       "<td>0.0045164</td>\n",
       "<td>1.1105507</td>\n",
       "<td>0.0022685</td>\n",
       "<td>0.0045299</td>\n",
       "<td>0.5578074</td>\n",
       "<td>0.5577424</td>\n",
       "<td>0.0004513</td>\n",
       "<td>0.9994956</td>\n",
       "<td>-99.5483632</td>\n",
       "<td>11.0550704</td>\n",
       "<td>0.1999028</td></tr>\n",
       "<tr><td>16</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0001672</td>\n",
       "<td>0.0050437</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0025333</td>\n",
       "<td>0.0026268</td>\n",
       "<td>0.50228</td>\n",
       "<td>0.5022308</td>\n",
       "<td>0.0005044</td>\n",
       "<td>1.0</td>\n",
       "<td>-99.4956332</td>\n",
       "<td>0.0</td>\n",
       "<td>0.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div></div>\n",
       "<div style='margin: 1em 0 1em 0;'><pre style='margin: 1em 0 1em 0;'>ModelMetricsBinomial: xgboost\n",
       "** Reported on validation data. **\n",
       "\n",
       "MSE: 0.038600168581279404\n",
       "RMSE: 0.1964692560714765\n",
       "LogLoss: 0.14264211397729937\n",
       "Mean Per-Class Error: 0.04874265748103474\n",
       "AUC: 0.9860367984758349\n",
       "AUCPR: 0.9841777019906762\n",
       "Gini: 0.9720735969516698</pre>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-24.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-24 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-24 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-24 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-24 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-24 .h2o-table th,\n",
       "#h2o-table-24 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-24 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-24\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.4899955771186135</caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>0</th>\n",
       "<th>1</th>\n",
       "<th>Error</th>\n",
       "<th>Rate</th></tr></thead>\n",
       "    <tbody><tr><td>0</td>\n",
       "<td>11844.0</td>\n",
       "<td>840.0</td>\n",
       "<td>0.0662</td>\n",
       "<td> (840.0/12684.0)</td></tr>\n",
       "<tr><td>1</td>\n",
       "<td>385.0</td>\n",
       "<td>11931.0</td>\n",
       "<td>0.0313</td>\n",
       "<td> (385.0/12316.0)</td></tr>\n",
       "<tr><td>Total</td>\n",
       "<td>12229.0</td>\n",
       "<td>12771.0</td>\n",
       "<td>0.049</td>\n",
       "<td> (1225.0/25000.0)</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-25.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-25 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-25 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-25 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-25 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-25 .h2o-table th,\n",
       "#h2o-table-25 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-25 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-25\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Maximum Metrics: Maximum metrics at their respective thresholds</caption>\n",
       "    <thead><tr><th>metric</th>\n",
       "<th>threshold</th>\n",
       "<th>value</th>\n",
       "<th>idx</th></tr></thead>\n",
       "    <tbody><tr><td>max f1</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9511699</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max f2</td>\n",
       "<td>0.2381639</td>\n",
       "<td>0.9684011</td>\n",
       "<td>281.0</td></tr>\n",
       "<tr><td>max f0point5</td>\n",
       "<td>0.7637601</td>\n",
       "<td>0.9521032</td>\n",
       "<td>144.0</td></tr>\n",
       "<tr><td>max accuracy</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.951</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max precision</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max recall</td>\n",
       "<td>0.0018354</td>\n",
       "<td>1.0</td>\n",
       "<td>398.0</td></tr>\n",
       "<tr><td>max specificity</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max absolute_mcc</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9026291</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max min_per_class_accuracy</td>\n",
       "<td>0.6303638</td>\n",
       "<td>0.9494966</td>\n",
       "<td>182.0</td></tr>\n",
       "<tr><td>max mean_per_class_accuracy</td>\n",
       "<td>0.4899956</td>\n",
       "<td>0.9512573</td>\n",
       "<td>217.0</td></tr>\n",
       "<tr><td>max tns</td>\n",
       "<td>0.9974683</td>\n",
       "<td>12684.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fns</td>\n",
       "<td>0.9974683</td>\n",
       "<td>12244.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fps</td>\n",
       "<td>0.0013435</td>\n",
       "<td>12684.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tps</td>\n",
       "<td>0.0018354</td>\n",
       "<td>12316.0</td>\n",
       "<td>398.0</td></tr>\n",
       "<tr><td>max tnr</td>\n",
       "<td>0.9974683</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fnr</td>\n",
       "<td>0.9974683</td>\n",
       "<td>0.9941539</td>\n",
       "<td>0.0</td></tr>\n",
       "<tr><td>max fpr</td>\n",
       "<td>0.0013435</td>\n",
       "<td>1.0</td>\n",
       "<td>399.0</td></tr>\n",
       "<tr><td>max tpr</td>\n",
       "<td>0.0018354</td>\n",
       "<td>1.0</td>\n",
       "<td>398.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-26.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-26 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-26 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-26 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-26 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-26 .h2o-table th,\n",
       "#h2o-table-26 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-26 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-26\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Gains/Lift Table: Avg response rate: 49.26 %, avg score: 49.47 %</caption>\n",
       "    <thead><tr><th>group</th>\n",
       "<th>cumulative_data_fraction</th>\n",
       "<th>lower_threshold</th>\n",
       "<th>lift</th>\n",
       "<th>cumulative_lift</th>\n",
       "<th>response_rate</th>\n",
       "<th>score</th>\n",
       "<th>cumulative_response_rate</th>\n",
       "<th>cumulative_score</th>\n",
       "<th>capture_rate</th>\n",
       "<th>cumulative_capture_rate</th>\n",
       "<th>gain</th>\n",
       "<th>cumulative_gain</th>\n",
       "<th>kolmogorov_smirnov</th></tr></thead>\n",
       "    <tbody><tr><td>1</td>\n",
       "<td>0.01028</td>\n",
       "<td>0.9961244</td>\n",
       "<td>2.0298798</td>\n",
       "<td>2.0298798</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967313</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9967313</td>\n",
       "<td>0.0208672</td>\n",
       "<td>0.0208672</td>\n",
       "<td>102.9879831</td>\n",
       "<td>102.9879831</td>\n",
       "<td>0.0208672</td></tr>\n",
       "<tr><td>2</td>\n",
       "<td>0.02032</td>\n",
       "<td>0.9958012</td>\n",
       "<td>2.0298798</td>\n",
       "<td>2.0298798</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9959175</td>\n",
       "<td>1.0</td>\n",
       "<td>0.9963292</td>\n",
       "<td>0.0203800</td>\n",
       "<td>0.0412472</td>\n",
       "<td>102.9879831</td>\n",
       "<td>102.9879831</td>\n",
       "<td>0.0412472</td></tr>\n",
       "<tr><td>3</td>\n",
       "<td>0.03016</td>\n",
       "<td>0.9954615</td>\n",
       "<td>2.0216283</td>\n",
       "<td>2.0271877</td>\n",
       "<td>0.9959350</td>\n",
       "<td>0.9956102</td>\n",
       "<td>0.9986737</td>\n",
       "<td>0.9960946</td>\n",
       "<td>0.0198928</td>\n",
       "<td>0.0611400</td>\n",
       "<td>102.1628287</td>\n",
       "<td>102.7187683</td>\n",
       "<td>0.0610611</td></tr>\n",
       "<tr><td>4</td>\n",
       "<td>0.04108</td>\n",
       "<td>0.9951944</td>\n",
       "<td>2.0224444</td>\n",
       "<td>2.0259268</td>\n",
       "<td>0.9963370</td>\n",
       "<td>0.9952887</td>\n",
       "<td>0.9980526</td>\n",
       "<td>0.9958804</td>\n",
       "<td>0.0220851</td>\n",
       "<td>0.0832251</td>\n",
       "<td>102.2444374</td>\n",
       "<td>102.5926803</td>\n",
       "<td>0.0830674</td></tr>\n",
       "<tr><td>5</td>\n",
       "<td>0.05256</td>\n",
       "<td>0.9949297</td>\n",
       "<td>2.0157343</td>\n",
       "<td>2.0237006</td>\n",
       "<td>0.9930314</td>\n",
       "<td>0.9950085</td>\n",
       "<td>0.9969559</td>\n",
       "<td>0.9956900</td>\n",
       "<td>0.0231406</td>\n",
       "<td>0.1063657</td>\n",
       "<td>101.5734327</td>\n",
       "<td>102.3700593</td>\n",
       "<td>0.1060503</td></tr>\n",
       "<tr><td>6</td>\n",
       "<td>0.1</td>\n",
       "<td>0.9932859</td>\n",
       "<td>2.0196106</td>\n",
       "<td>2.0217603</td>\n",
       "<td>0.9949410</td>\n",
       "<td>0.9941552</td>\n",
       "<td>0.996</td>\n",
       "<td>0.9949619</td>\n",
       "<td>0.0958103</td>\n",
       "<td>0.2021760</td>\n",
       "<td>101.9610625</td>\n",
       "<td>102.1760312</td>\n",
       "<td>0.2013876</td></tr>\n",
       "<tr><td>7</td>\n",
       "<td>0.15</td>\n",
       "<td>0.9902694</td>\n",
       "<td>2.0233842</td>\n",
       "<td>2.0223016</td>\n",
       "<td>0.9968</td>\n",
       "<td>0.9919490</td>\n",
       "<td>0.9962667</td>\n",
       "<td>0.9939576</td>\n",
       "<td>0.1011692</td>\n",
       "<td>0.3033452</td>\n",
       "<td>102.3384216</td>\n",
       "<td>102.2301613</td>\n",
       "<td>0.3022415</td></tr>\n",
       "<tr><td>8</td>\n",
       "<td>0.2</td>\n",
       "<td>0.9840660</td>\n",
       "<td>2.0136408</td>\n",
       "<td>2.0201364</td>\n",
       "<td>0.992</td>\n",
       "<td>0.9873945</td>\n",
       "<td>0.9952</td>\n",
       "<td>0.9923168</td>\n",
       "<td>0.1006820</td>\n",
       "<td>0.4040273</td>\n",
       "<td>101.3640792</td>\n",
       "<td>102.0136408</td>\n",
       "<td>0.4021351</td></tr>\n",
       "<tr><td>9</td>\n",
       "<td>0.3</td>\n",
       "<td>0.9523897</td>\n",
       "<td>1.9852225</td>\n",
       "<td>2.0084984</td>\n",
       "<td>0.978</td>\n",
       "<td>0.9717051</td>\n",
       "<td>0.9894667</td>\n",
       "<td>0.9854462</td>\n",
       "<td>0.1985222</td>\n",
       "<td>0.6025495</td>\n",
       "<td>98.5222475</td>\n",
       "<td>100.8498430</td>\n",
       "<td>0.5963212</td></tr>\n",
       "<tr><td>10</td>\n",
       "<td>0.4</td>\n",
       "<td>0.8671057</td>\n",
       "<td>1.9235141</td>\n",
       "<td>1.9872524</td>\n",
       "<td>0.9476</td>\n",
       "<td>0.9142567</td>\n",
       "<td>0.979</td>\n",
       "<td>0.9676488</td>\n",
       "<td>0.1923514</td>\n",
       "<td>0.7949009</td>\n",
       "<td>92.3514128</td>\n",
       "<td>98.7252355</td>\n",
       "<td>0.7783447</td></tr>\n",
       "<tr><td>11</td>\n",
       "<td>0.5</td>\n",
       "<td>0.5822353</td>\n",
       "<td>1.6174082</td>\n",
       "<td>1.9132835</td>\n",
       "<td>0.7968</td>\n",
       "<td>0.7825059</td>\n",
       "<td>0.94256</td>\n",
       "<td>0.9306202</td>\n",
       "<td>0.1617408</td>\n",
       "<td>0.9566418</td>\n",
       "<td>61.7408249</td>\n",
       "<td>91.3283534</td>\n",
       "<td>0.9000350</td></tr>\n",
       "<tr><td>12</td>\n",
       "<td>0.6</td>\n",
       "<td>0.0667765</td>\n",
       "<td>0.3775576</td>\n",
       "<td>1.6573292</td>\n",
       "<td>0.186</td>\n",
       "<td>0.2480180</td>\n",
       "<td>0.8164667</td>\n",
       "<td>0.8168532</td>\n",
       "<td>0.0377558</td>\n",
       "<td>0.9943975</td>\n",
       "<td>-62.2442351</td>\n",
       "<td>65.7329219</td>\n",
       "<td>0.7773524</td></tr>\n",
       "<tr><td>13</td>\n",
       "<td>0.7</td>\n",
       "<td>0.0125771</td>\n",
       "<td>0.0186749</td>\n",
       "<td>1.4232357</td>\n",
       "<td>0.0092</td>\n",
       "<td>0.0304482</td>\n",
       "<td>0.7011429</td>\n",
       "<td>0.7045096</td>\n",
       "<td>0.0018675</td>\n",
       "<td>0.9962650</td>\n",
       "<td>-98.1325106</td>\n",
       "<td>42.3235744</td>\n",
       "<td>0.5839345</td></tr>\n",
       "<tr><td>14</td>\n",
       "<td>0.80048</td>\n",
       "<td>0.0057412</td>\n",
       "<td>0.0185857</td>\n",
       "<td>1.2469175</td>\n",
       "<td>0.0091561</td>\n",
       "<td>0.0081574</td>\n",
       "<td>0.6142814</td>\n",
       "<td>0.6171002</td>\n",
       "<td>0.0018675</td>\n",
       "<td>0.9981325</td>\n",
       "<td>-98.1414317</td>\n",
       "<td>24.6917488</td>\n",
       "<td>0.3895705</td></tr>\n",
       "<tr><td>15</td>\n",
       "<td>0.9014</td>\n",
       "<td>0.0034592</td>\n",
       "<td>0.0104592</td>\n",
       "<td>1.1084846</td>\n",
       "<td>0.0051526</td>\n",
       "<td>0.0045211</td>\n",
       "<td>0.5460839</td>\n",
       "<td>0.5485164</td>\n",
       "<td>0.0010555</td>\n",
       "<td>0.9991880</td>\n",
       "<td>-98.9540849</td>\n",
       "<td>10.8484633</td>\n",
       "<td>0.1927390</td></tr>\n",
       "<tr><td>16</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0002609</td>\n",
       "<td>0.0082348</td>\n",
       "<td>1.0</td>\n",
       "<td>0.0040568</td>\n",
       "<td>0.0026446</td>\n",
       "<td>0.49264</td>\n",
       "<td>0.4946934</td>\n",
       "<td>0.0008120</td>\n",
       "<td>1.0</td>\n",
       "<td>-99.1765193</td>\n",
       "<td>0.0</td>\n",
       "<td>0.0</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div></div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-27.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-27 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-27 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-27 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-27 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-27 .h2o-table th,\n",
       "#h2o-table-27 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-27 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-27\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Scoring History: </caption>\n",
       "    <thead><tr><th></th>\n",
       "<th>timestamp</th>\n",
       "<th>duration</th>\n",
       "<th>number_of_trees</th>\n",
       "<th>training_rmse</th>\n",
       "<th>training_logloss</th>\n",
       "<th>training_auc</th>\n",
       "<th>training_pr_auc</th>\n",
       "<th>training_lift</th>\n",
       "<th>training_classification_error</th>\n",
       "<th>training_custom</th>\n",
       "<th>validation_rmse</th>\n",
       "<th>validation_logloss</th>\n",
       "<th>validation_auc</th>\n",
       "<th>validation_pr_auc</th>\n",
       "<th>validation_lift</th>\n",
       "<th>validation_classification_error</th>\n",
       "<th>validation_custom</th></tr></thead>\n",
       "    <tbody><tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.030 sec</td>\n",
       "<td>0.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.693147</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.693147</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.118 sec</td>\n",
       "<td>1.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.508456</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.509577</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.127 sec</td>\n",
       "<td>2.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.403205</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.404315</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.135 sec</td>\n",
       "<td>3.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.333565</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.335569</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.144 sec</td>\n",
       "<td>4.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.286705</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.28943</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.151 sec</td>\n",
       "<td>5.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.254899</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.257526</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.161 sec</td>\n",
       "<td>6.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.232886</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.235729</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.169 sec</td>\n",
       "<td>7.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.217304</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.220206</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.177 sec</td>\n",
       "<td>8.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.200464</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.204054</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.185 sec</td>\n",
       "<td>9.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.190888</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.195105</td></tr>\n",
       "<tr><td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td>\n",
       "<td>---</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.224 sec</td>\n",
       "<td>13.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.161882</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.168298</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.234 sec</td>\n",
       "<td>14.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.158723</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.165693</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.243 sec</td>\n",
       "<td>15.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.155611</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.163391</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.251 sec</td>\n",
       "<td>16.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.151794</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.160778</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.261 sec</td>\n",
       "<td>17.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.147082</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.156551</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.270 sec</td>\n",
       "<td>18.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.140969</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.151666</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.279 sec</td>\n",
       "<td>19.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.139175</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.150583</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.291 sec</td>\n",
       "<td>20.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.136121</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.148415</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.301 sec</td>\n",
       "<td>21.0</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.131936</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>nan</td>\n",
       "<td>0.144895</td></tr>\n",
       "<tr><td></td>\n",
       "<td>2022-10-28 14:19:33</td>\n",
       "<td> 0.310 sec</td>\n",
       "<td>22.0</td>\n",
       "<td>0.1864343</td>\n",
       "<td>0.1290141</td>\n",
       "<td>0.9893340</td>\n",
       "<td>0.9887041</td>\n",
       "<td>1.9909214</td>\n",
       "<td>0.0438133</td>\n",
       "<td>0.129014</td>\n",
       "<td>0.1964693</td>\n",
       "<td>0.1426421</td>\n",
       "<td>0.9860368</td>\n",
       "<td>0.9841777</td>\n",
       "<td>2.0298798</td>\n",
       "<td>0.049</td>\n",
       "<td>0.142642</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "<pre style='font-size: smaller; margin-bottom: 1em;'>[23 rows x 18 columns]</pre></div>\n",
       "<div style='margin: 1em 0 1em 0;'>\n",
       "<style>\n",
       "\n",
       "#h2o-table-28.h2o-container {\n",
       "  overflow-x: auto;\n",
       "}\n",
       "#h2o-table-28 .h2o-table {\n",
       "  /* width: 100%; */\n",
       "  margin-top: 1em;\n",
       "  margin-bottom: 1em;\n",
       "}\n",
       "#h2o-table-28 .h2o-table caption {\n",
       "  white-space: nowrap;\n",
       "  caption-side: top;\n",
       "  text-align: left;\n",
       "  /* margin-left: 1em; */\n",
       "  margin: 0;\n",
       "  font-size: larger;\n",
       "}\n",
       "#h2o-table-28 .h2o-table thead {\n",
       "  white-space: nowrap; \n",
       "  position: sticky;\n",
       "  top: 0;\n",
       "  box-shadow: 0 -1px inset;\n",
       "}\n",
       "#h2o-table-28 .h2o-table tbody {\n",
       "  overflow: auto;\n",
       "}\n",
       "#h2o-table-28 .h2o-table th,\n",
       "#h2o-table-28 .h2o-table td {\n",
       "  text-align: right;\n",
       "  /* border: 1px solid; */\n",
       "}\n",
       "#h2o-table-28 .h2o-table tr:nth-child(even) {\n",
       "  /* background: #F5F5F5 */\n",
       "}\n",
       "\n",
       "</style>      \n",
       "<div id=\"h2o-table-28\" class=\"h2o-container\">\n",
       "  <table class=\"h2o-table\">\n",
       "    <caption>Variable Importances: </caption>\n",
       "    <thead><tr><th>variable</th>\n",
       "<th>relative_importance</th>\n",
       "<th>scaled_importance</th>\n",
       "<th>percentage</th></tr></thead>\n",
       "    <tbody><tr><td>C11</td>\n",
       "<td>70178.75</td>\n",
       "<td>1.0</td>\n",
       "<td>0.4261358</td></tr>\n",
       "<tr><td>C2</td>\n",
       "<td>35732.6640625</td>\n",
       "<td>0.5091664</td>\n",
       "<td>0.2169741</td></tr>\n",
       "<tr><td>C13</td>\n",
       "<td>21608.4863281</td>\n",
       "<td>0.3079064</td>\n",
       "<td>0.1312100</td></tr>\n",
       "<tr><td>C8</td>\n",
       "<td>15654.5498047</td>\n",
       "<td>0.2230668</td>\n",
       "<td>0.0950568</td></tr>\n",
       "<tr><td>C3</td>\n",
       "<td>12402.2607422</td>\n",
       "<td>0.1767239</td>\n",
       "<td>0.0753084</td></tr>\n",
       "<tr><td>C19</td>\n",
       "<td>5739.7641602</td>\n",
       "<td>0.0817878</td>\n",
       "<td>0.0348527</td></tr>\n",
       "<tr><td>C5</td>\n",
       "<td>2835.9191895</td>\n",
       "<td>0.0404099</td>\n",
       "<td>0.0172201</td></tr>\n",
       "<tr><td>C6</td>\n",
       "<td>88.9352493</td>\n",
       "<td>0.0012673</td>\n",
       "<td>0.0005400</td></tr>\n",
       "<tr><td>C12</td>\n",
       "<td>51.9876480</td>\n",
       "<td>0.0007408</td>\n",
       "<td>0.0003157</td></tr>\n",
       "<tr><td>C17</td>\n",
       "<td>49.1711044</td>\n",
       "<td>0.0007007</td>\n",
       "<td>0.0002986</td></tr>\n",
       "<tr><td>C7</td>\n",
       "<td>48.0521317</td>\n",
       "<td>0.0006847</td>\n",
       "<td>0.0002918</td></tr>\n",
       "<tr><td>C14</td>\n",
       "<td>46.5472183</td>\n",
       "<td>0.0006633</td>\n",
       "<td>0.0002826</td></tr>\n",
       "<tr><td>C10</td>\n",
       "<td>43.8788605</td>\n",
       "<td>0.0006252</td>\n",
       "<td>0.0002664</td></tr>\n",
       "<tr><td>C4</td>\n",
       "<td>42.4346542</td>\n",
       "<td>0.0006047</td>\n",
       "<td>0.0002577</td></tr>\n",
       "<tr><td>C18</td>\n",
       "<td>39.3952980</td>\n",
       "<td>0.0005614</td>\n",
       "<td>0.0002392</td></tr>\n",
       "<tr><td>C1</td>\n",
       "<td>32.5539932</td>\n",
       "<td>0.0004639</td>\n",
       "<td>0.0001977</td></tr>\n",
       "<tr><td>C15</td>\n",
       "<td>25.9528160</td>\n",
       "<td>0.0003698</td>\n",
       "<td>0.0001576</td></tr>\n",
       "<tr><td>C20</td>\n",
       "<td>23.1192322</td>\n",
       "<td>0.0003294</td>\n",
       "<td>0.0001404</td></tr>\n",
       "<tr><td>C16</td>\n",
       "<td>22.9983807</td>\n",
       "<td>0.0003277</td>\n",
       "<td>0.0001396</td></tr>\n",
       "<tr><td>C9</td>\n",
       "<td>18.9113159</td>\n",
       "<td>0.0002695</td>\n",
       "<td>0.0001148</td></tr></tbody>\n",
       "  </table>\n",
       "</div>\n",
       "</div><pre style=\"font-size: smaller; margin: 1em 0 0 0;\">\n",
       "\n",
       "[tips]\n",
       "Use `model.explain()` to inspect the model.\n",
       "--\n",
       "Use `h2o.display.toggle_user_tips()` to switch on/off this section.</pre>"
      ],
      "text/plain": [
       "Model Details\n",
       "=============\n",
       "H2OXGBoostEstimator : XGBoost\n",
       "Model Key: XGBoost_model_python_1666962310297_616\n",
       "\n",
       "\n",
       "Model Summary: \n",
       "    number_of_trees\n",
       "--  -----------------\n",
       "    22\n",
       "\n",
       "ModelMetricsBinomial: xgboost\n",
       "** Reported on train data. **\n",
       "\n",
       "MSE: 0.03475774769265435\n",
       "RMSE: 0.18643429859511995\n",
       "LogLoss: 0.12901406890397824\n",
       "Mean Per-Class Error: 0.04387881533186782\n",
       "AUC: 0.9893340459497513\n",
       "AUCPR: 0.988704085388505\n",
       "Gini: 0.9786680918995025\n",
       "\n",
       "Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.5101676438305829\n",
       "       0      1      Error    Rate\n",
       "-----  -----  -----  -------  ----------------\n",
       "0      35155  2174   0.0582   (2174.0/37329.0)\n",
       "1      1112   36559  0.0295   (1112.0/37671.0)\n",
       "Total  36267  38733  0.0438   (3286.0/75000.0)\n",
       "\n",
       "Maximum Metrics: Maximum metrics at their respective thresholds\n",
       "metric                       threshold    value     idx\n",
       "---------------------------  -----------  --------  -----\n",
       "max f1                       0.510168     0.956992  214\n",
       "max f2                       0.308281     0.971508  263\n",
       "max f0point5                 0.753261     0.959208  147\n",
       "max accuracy                 0.556171     0.9564    201\n",
       "max precision                0.997456     1         0\n",
       "max recall                   0.00160282   1         399\n",
       "max specificity              0.997456     1         0\n",
       "max absolute_mcc             0.556171     0.912928  201\n",
       "max min_per_class_accuracy   0.623985     0.954807  184\n",
       "max mean_per_class_accuracy  0.556171     0.95636   201\n",
       "max tns                      0.997456     37329     0\n",
       "max fns                      0.997456     37460     0\n",
       "max fps                      0.00160282   37329     399\n",
       "max tps                      0.00160282   37671     399\n",
       "max tnr                      0.997456     1         0\n",
       "max fnr                      0.997456     0.994399  0\n",
       "max fpr                      0.00160282   1         399\n",
       "max tpr                      0.00160282   1         399\n",
       "\n",
       "Gains/Lift Table: Avg response rate: 50.23 %, avg score: 50.22 %\n",
       "group    cumulative_data_fraction    lower_threshold    lift        cumulative_lift    response_rate    score       cumulative_response_rate    cumulative_score    capture_rate    cumulative_capture_rate    gain      cumulative_gain    kolmogorov_smirnov\n",
       "-------  --------------------------  -----------------  ----------  -----------------  ---------------  ----------  --------------------------  ------------------  --------------  -------------------------  --------  -----------------  --------------------\n",
       "1        0.0101733                   0.99611            1.99092     1.99092            1                0.996752    1                           0.996752            0.0202543       0.0202543                  99.0921   99.0921            0.0202543\n",
       "2        0.0200267                   0.995715           1.99092     1.99092            1                0.995889    1                           0.996327            0.0196172       0.0398715                  99.0921   99.0921            0.0398715\n",
       "3        0.03068                     0.995396           1.99092     1.99092            1                0.995549    1                           0.996057            0.0212099       0.0610815                  99.0921   99.0921            0.0610815\n",
       "4        0.0401067                   0.995194           1.98811     1.99026            0.998586         0.995271    0.999668                    0.995872            0.0187412       0.0798227                  98.8105   99.026             0.0797959\n",
       "5        0.0517733                   0.99493            1.98637     1.98938            0.997714         0.995008    0.999227                    0.995678            0.0231743       0.102997                   98.6371   98.9383            0.102917\n",
       "6        0.100013                    0.99331            1.98817     1.9888             0.998618         0.994166    0.998933                    0.994948            0.0959093       0.198906                   98.817    98.8798            0.198692\n",
       "7        0.15008                     0.990498           1.98403     1.98721            0.996538         0.992058    0.998134                    0.993984            0.0993337       0.29824                    98.4029   98.7207            0.297677\n",
       "8        0.200013                    0.98496            1.98242     1.98601            0.995728         0.987913    0.997533                    0.992468            0.0989886       0.397229                   98.2415   98.6011            0.396237\n",
       "9        0.30004                     0.956105           1.9673      1.97977            0.988136         0.974039    0.994401                    0.986324            0.196783        0.594011                   96.7302   97.9774            0.590636\n",
       "10       0.400013                    0.877061           1.91233     1.96292            0.960523         0.921008    0.985934                    0.97                0.191182        0.785193                   91.2325   96.2917            0.773888\n",
       "11       0.500013                    0.638348           1.67636     1.90561            0.842            0.805119    0.957148                    0.937025            0.167636        0.952828                   67.6356   90.5606            0.909779\n",
       "12       0.6                         0.0781245          0.426114    1.65906            0.214029         0.287618    0.833311                    0.828805            0.0426057       0.995434                   -57.3886  65.9057            0.794491\n",
       "13       0.700013                    0.0128222          0.0238879   1.42543            0.0119984        0.0339001   0.715967                    0.715234            0.00238911      0.997823                   -97.6112  42.5435            0.598348\n",
       "14       0.80008                     0.00579305         0.0122028   1.24868            0.00612925       0.00841651  0.627187                    0.626832            0.0012211       0.999044                   -98.7797  24.8681            0.399752\n",
       "15       0.9                         0.00344276         0.00451637  1.11055            0.00226848       0.00452987  0.557807                    0.557742            0.000451276     0.999496                   -99.5484  11.0551            0.199903\n",
       "16       1                           0.000167174        0.00504367  1                  0.00253333       0.00262682  0.50228                     0.502231            0.000504367     1                          -99.4956  0                  0\n",
       "\n",
       "ModelMetricsBinomial: xgboost\n",
       "** Reported on validation data. **\n",
       "\n",
       "MSE: 0.038600168581279404\n",
       "RMSE: 0.1964692560714765\n",
       "LogLoss: 0.14264211397729937\n",
       "Mean Per-Class Error: 0.04874265748103474\n",
       "AUC: 0.9860367984758349\n",
       "AUCPR: 0.9841777019906762\n",
       "Gini: 0.9720735969516698\n",
       "\n",
       "Confusion Matrix (Act/Pred) for max f1 @ threshold = 0.4899955771186135\n",
       "       0      1      Error    Rate\n",
       "-----  -----  -----  -------  ----------------\n",
       "0      11844  840    0.0662   (840.0/12684.0)\n",
       "1      385    11931  0.0313   (385.0/12316.0)\n",
       "Total  12229  12771  0.049    (1225.0/25000.0)\n",
       "\n",
       "Maximum Metrics: Maximum metrics at their respective thresholds\n",
       "metric                       threshold    value     idx\n",
       "---------------------------  -----------  --------  -----\n",
       "max f1                       0.489996     0.95117   217\n",
       "max f2                       0.238164     0.968401  281\n",
       "max f0point5                 0.76376      0.952103  144\n",
       "max accuracy                 0.489996     0.951     217\n",
       "max precision                0.997468     1         0\n",
       "max recall                   0.00183535   1         398\n",
       "max specificity              0.997468     1         0\n",
       "max absolute_mcc             0.489996     0.902629  217\n",
       "max min_per_class_accuracy   0.630364     0.949497  182\n",
       "max mean_per_class_accuracy  0.489996     0.951257  217\n",
       "max tns                      0.997468     12684     0\n",
       "max fns                      0.997468     12244     0\n",
       "max fps                      0.00134346   12684     399\n",
       "max tps                      0.00183535   12316     398\n",
       "max tnr                      0.997468     1         0\n",
       "max fnr                      0.997468     0.994154  0\n",
       "max fpr                      0.00134346   1         399\n",
       "max tpr                      0.00183535   1         398\n",
       "\n",
       "Gains/Lift Table: Avg response rate: 49.26 %, avg score: 49.47 %\n",
       "group    cumulative_data_fraction    lower_threshold    lift        cumulative_lift    response_rate    score       cumulative_response_rate    cumulative_score    capture_rate    cumulative_capture_rate    gain      cumulative_gain    kolmogorov_smirnov\n",
       "-------  --------------------------  -----------------  ----------  -----------------  ---------------  ----------  --------------------------  ------------------  --------------  -------------------------  --------  -----------------  --------------------\n",
       "1        0.01028                     0.996124           2.02988     2.02988            1                0.996731    1                           0.996731            0.0208672       0.0208672                  102.988   102.988            0.0208672\n",
       "2        0.02032                     0.995801           2.02988     2.02988            1                0.995918    1                           0.996329            0.02038         0.0412472                  102.988   102.988            0.0412472\n",
       "3        0.03016                     0.995462           2.02163     2.02719            0.995935         0.99561     0.998674                    0.996095            0.0198928       0.06114                    102.163   102.719            0.0610611\n",
       "4        0.04108                     0.995194           2.02244     2.02593            0.996337         0.995289    0.998053                    0.99588             0.0220851       0.0832251                  102.244   102.593            0.0830674\n",
       "5        0.05256                     0.99493            2.01573     2.0237             0.993031         0.995009    0.996956                    0.99569             0.0231406       0.106366                   101.573   102.37             0.10605\n",
       "6        0.1                         0.993286           2.01961     2.02176            0.994941         0.994155    0.996                       0.994962            0.0958103       0.202176                   101.961   102.176            0.201388\n",
       "7        0.15                        0.990269           2.02338     2.0223             0.9968           0.991949    0.996267                    0.993958            0.101169        0.303345                   102.338   102.23             0.302241\n",
       "8        0.2                         0.984066           2.01364     2.02014            0.992            0.987394    0.9952                      0.992317            0.100682        0.404027                   101.364   102.014            0.402135\n",
       "9        0.3                         0.95239            1.98522     2.0085             0.978            0.971705    0.989467                    0.985446            0.198522        0.60255                    98.5222   100.85             0.596321\n",
       "10       0.4                         0.867106           1.92351     1.98725            0.9476           0.914257    0.979                       0.967649            0.192351        0.794901                   92.3514   98.7252            0.778345\n",
       "11       0.5                         0.582235           1.61741     1.91328            0.7968           0.782506    0.94256                     0.93062             0.161741        0.956642                   61.7408   91.3284            0.900035\n",
       "12       0.6                         0.0667765          0.377558    1.65733            0.186            0.248018    0.816467                    0.816853            0.0377558       0.994398                   -62.2442  65.7329            0.777352\n",
       "13       0.7                         0.0125771          0.0186749   1.42324            0.0092           0.0304482   0.701143                    0.70451             0.00186749      0.996265                   -98.1325  42.3236            0.583935\n",
       "14       0.80048                     0.00574124         0.0185857   1.24692            0.00915605       0.00815736  0.614281                    0.6171              0.00186749      0.998133                   -98.1414  24.6917            0.389571\n",
       "15       0.9014                      0.00345923         0.0104592   1.10848            0.0051526        0.0045211   0.546084                    0.548516            0.00105554      0.999188                   -98.9541  10.8485            0.192739\n",
       "16       1                           0.000260908        0.00823481  1                  0.0040568        0.00264459  0.49264                     0.494693            0.000811952     1                          -99.1765  0                  0\n",
       "\n",
       "Scoring History: \n",
       "     timestamp            duration    number_of_trees    training_rmse        training_logloss     training_auc        training_pr_auc    training_lift       training_classification_error    training_custom    validation_rmse     validation_logloss    validation_auc      validation_pr_auc    validation_lift    validation_classification_error    validation_custom\n",
       "---  -------------------  ----------  -----------------  -------------------  -------------------  ------------------  -----------------  ------------------  -------------------------------  -----------------  ------------------  --------------------  ------------------  -------------------  -----------------  ---------------------------------  -------------------\n",
       "     2022-10-28 14:19:33  0.030 sec   0.0                nan                  nan                  nan                 nan                nan                 nan                              0.693147           nan                 nan                   nan                 nan                  nan                nan                                0.693147\n",
       "     2022-10-28 14:19:33  0.118 sec   1.0                nan                  nan                  nan                 nan                nan                 nan                              0.508456           nan                 nan                   nan                 nan                  nan                nan                                0.509577\n",
       "     2022-10-28 14:19:33  0.127 sec   2.0                nan                  nan                  nan                 nan                nan                 nan                              0.403205           nan                 nan                   nan                 nan                  nan                nan                                0.404315\n",
       "     2022-10-28 14:19:33  0.135 sec   3.0                nan                  nan                  nan                 nan                nan                 nan                              0.333565           nan                 nan                   nan                 nan                  nan                nan                                0.335569\n",
       "     2022-10-28 14:19:33  0.144 sec   4.0                nan                  nan                  nan                 nan                nan                 nan                              0.286705           nan                 nan                   nan                 nan                  nan                nan                                0.28943\n",
       "     2022-10-28 14:19:33  0.151 sec   5.0                nan                  nan                  nan                 nan                nan                 nan                              0.254899           nan                 nan                   nan                 nan                  nan                nan                                0.257526\n",
       "     2022-10-28 14:19:33  0.161 sec   6.0                nan                  nan                  nan                 nan                nan                 nan                              0.232886           nan                 nan                   nan                 nan                  nan                nan                                0.235729\n",
       "     2022-10-28 14:19:33  0.169 sec   7.0                nan                  nan                  nan                 nan                nan                 nan                              0.217304           nan                 nan                   nan                 nan                  nan                nan                                0.220206\n",
       "     2022-10-28 14:19:33  0.177 sec   8.0                nan                  nan                  nan                 nan                nan                 nan                              0.200464           nan                 nan                   nan                 nan                  nan                nan                                0.204054\n",
       "     2022-10-28 14:19:33  0.185 sec   9.0                nan                  nan                  nan                 nan                nan                 nan                              0.190888           nan                 nan                   nan                 nan                  nan                nan                                0.195105\n",
       "---  ---                  ---         ---                ---                  ---                  ---                 ---                ---                 ---                              ---                ---                 ---                   ---                 ---                  ---                ---                                ---\n",
       "     2022-10-28 14:19:33  0.224 sec   13.0               nan                  nan                  nan                 nan                nan                 nan                              0.161882           nan                 nan                   nan                 nan                  nan                nan                                0.168298\n",
       "     2022-10-28 14:19:33  0.234 sec   14.0               nan                  nan                  nan                 nan                nan                 nan                              0.158723           nan                 nan                   nan                 nan                  nan                nan                                0.165693\n",
       "     2022-10-28 14:19:33  0.243 sec   15.0               nan                  nan                  nan                 nan                nan                 nan                              0.155611           nan                 nan                   nan                 nan                  nan                nan                                0.163391\n",
       "     2022-10-28 14:19:33  0.251 sec   16.0               nan                  nan                  nan                 nan                nan                 nan                              0.151794           nan                 nan                   nan                 nan                  nan                nan                                0.160778\n",
       "     2022-10-28 14:19:33  0.261 sec   17.0               nan                  nan                  nan                 nan                nan                 nan                              0.147082           nan                 nan                   nan                 nan                  nan                nan                                0.156551\n",
       "     2022-10-28 14:19:33  0.270 sec   18.0               nan                  nan                  nan                 nan                nan                 nan                              0.140969           nan                 nan                   nan                 nan                  nan                nan                                0.151666\n",
       "     2022-10-28 14:19:33  0.279 sec   19.0               nan                  nan                  nan                 nan                nan                 nan                              0.139175           nan                 nan                   nan                 nan                  nan                nan                                0.150583\n",
       "     2022-10-28 14:19:33  0.291 sec   20.0               nan                  nan                  nan                 nan                nan                 nan                              0.136121           nan                 nan                   nan                 nan                  nan                nan                                0.148415\n",
       "     2022-10-28 14:19:33  0.301 sec   21.0               nan                  nan                  nan                 nan                nan                 nan                              0.131936           nan                 nan                   nan                 nan                  nan                nan                                0.144895\n",
       "     2022-10-28 14:19:33  0.310 sec   22.0               0.18643429859511995  0.12901406890397824  0.9893340459497513  0.988704085388505  1.9909213984231904  0.043813333333333336             0.129014           0.1964692560714765  0.14264211397729937   0.9860367984758349  0.9841777019906762   2.029879831113998  0.049                              0.142642\n",
       "[23 rows x 18 columns]\n",
       "\n",
       "\n",
       "Variable Importances: \n",
       "variable    relative_importance    scaled_importance    percentage\n",
       "----------  ---------------------  -------------------  ------------\n",
       "C11         70178.8                1                    0.426136\n",
       "C2          35732.7                0.509166             0.216974\n",
       "C13         21608.5                0.307906             0.13121\n",
       "C8          15654.5                0.223067             0.0950568\n",
       "C3          12402.3                0.176724             0.0753084\n",
       "C19         5739.76                0.0817878            0.0348527\n",
       "C5          2835.92                0.0404099            0.0172201\n",
       "C6          88.9352                0.00126727           0.000540028\n",
       "C12         51.9876                0.000740789          0.000315677\n",
       "C17         49.1711                0.000700655          0.000298574\n",
       "C7          48.0521                0.000684711          0.00029178\n",
       "C14         46.5472                0.000663267          0.000282642\n",
       "C10         43.8789                0.000625244          0.000266439\n",
       "C4          42.4347                0.000604665          0.00025767\n",
       "C18         39.3953                0.000561357          0.000239214\n",
       "C1          32.554                 0.000463873          0.000197673\n",
       "C15         25.9528                0.00036981           0.000157589\n",
       "C20         23.1192                0.000329434          0.000140383\n",
       "C16         22.9984                0.000327711          0.00013965\n",
       "C9          18.9113                0.000269474          0.000114832\n",
       "\n",
       "[tips]\n",
       "Use `model.explain()` to inspect the model.\n",
       "--\n",
       "Use `h2o.display.toggle_user_tips()` to switch on/off this section."
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_eval_only = H2OXGBoostEstimator(ntrees=1000, max_depth=6, score_each_iteration=True, \n",
    "                                      eval_metric=\"logloss\",\n",
    "                                      stopping_rounds=3, stopping_tolerance=0.05, stopping_metric=\"custom\",\n",
    "                                      score_eval_metric_only=True)\n",
    "model_eval_only.train(y=\"y\", training_frame=train, validation_frame=valid)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5967ea3d",
   "metadata": {},
   "source": [
    "Scoring history will show undefined values for H2O metrics for all scoring iteration except for the final one. Values of columns `training_custom` and `validation_custom` will be the only ones populated for all of the iterations.\n",
    "\n",
    "In the final iteration H2O performs full scoring, that is why we see all values defined in the last row of the scoring history."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "7d99e876",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th>timestamp</th>\n",
       "      <th>duration</th>\n",
       "      <th>number_of_trees</th>\n",
       "      <th>training_rmse</th>\n",
       "      <th>training_logloss</th>\n",
       "      <th>training_auc</th>\n",
       "      <th>training_pr_auc</th>\n",
       "      <th>training_lift</th>\n",
       "      <th>training_classification_error</th>\n",
       "      <th>training_custom</th>\n",
       "      <th>validation_rmse</th>\n",
       "      <th>validation_logloss</th>\n",
       "      <th>validation_auc</th>\n",
       "      <th>validation_pr_auc</th>\n",
       "      <th>validation_lift</th>\n",
       "      <th>validation_classification_error</th>\n",
       "      <th>validation_custom</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.030 sec</td>\n",
       "      <td>0.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.693147</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.693147</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.118 sec</td>\n",
       "      <td>1.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.508456</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.509577</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.127 sec</td>\n",
       "      <td>2.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.403205</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.404315</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.135 sec</td>\n",
       "      <td>3.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.333565</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.335569</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.144 sec</td>\n",
       "      <td>4.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.286705</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.289430</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.151 sec</td>\n",
       "      <td>5.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.254899</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.257526</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>6</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.161 sec</td>\n",
       "      <td>6.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.232886</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.235729</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>7</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.169 sec</td>\n",
       "      <td>7.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.217304</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.220206</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>8</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.177 sec</td>\n",
       "      <td>8.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.200464</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.204054</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>9</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.185 sec</td>\n",
       "      <td>9.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.190888</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.195105</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.194 sec</td>\n",
       "      <td>10.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.182967</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.187705</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>11</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.202 sec</td>\n",
       "      <td>11.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.173463</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.178748</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>12</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.213 sec</td>\n",
       "      <td>12.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.167261</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.172957</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>13</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.224 sec</td>\n",
       "      <td>13.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.161882</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.168298</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>14</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.234 sec</td>\n",
       "      <td>14.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.158723</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.165693</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.243 sec</td>\n",
       "      <td>15.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.155611</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.163391</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>16</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.251 sec</td>\n",
       "      <td>16.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.151794</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.160778</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>17</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.261 sec</td>\n",
       "      <td>17.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.147082</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.156551</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>18</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.270 sec</td>\n",
       "      <td>18.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.140969</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.151666</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>19</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.279 sec</td>\n",
       "      <td>19.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.139175</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.150583</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.291 sec</td>\n",
       "      <td>20.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.136121</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.148415</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>21</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.301 sec</td>\n",
       "      <td>21.0</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.131936</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>NaN</td>\n",
       "      <td>0.144895</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>22</th>\n",
       "      <td></td>\n",
       "      <td>2022-10-28 14:19:33</td>\n",
       "      <td>0.310 sec</td>\n",
       "      <td>22.0</td>\n",
       "      <td>0.186434</td>\n",
       "      <td>0.129014</td>\n",
       "      <td>0.989334</td>\n",
       "      <td>0.988704</td>\n",
       "      <td>1.990921</td>\n",
       "      <td>0.043813</td>\n",
       "      <td>0.129014</td>\n",
       "      <td>0.196469</td>\n",
       "      <td>0.142642</td>\n",
       "      <td>0.986037</td>\n",
       "      <td>0.984178</td>\n",
       "      <td>2.02988</td>\n",
       "      <td>0.049</td>\n",
       "      <td>0.142642</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                timestamp    duration  number_of_trees  training_rmse  \\\n",
       "0     2022-10-28 14:19:33   0.030 sec              0.0            NaN   \n",
       "1     2022-10-28 14:19:33   0.118 sec              1.0            NaN   \n",
       "2     2022-10-28 14:19:33   0.127 sec              2.0            NaN   \n",
       "3     2022-10-28 14:19:33   0.135 sec              3.0            NaN   \n",
       "4     2022-10-28 14:19:33   0.144 sec              4.0            NaN   \n",
       "5     2022-10-28 14:19:33   0.151 sec              5.0            NaN   \n",
       "6     2022-10-28 14:19:33   0.161 sec              6.0            NaN   \n",
       "7     2022-10-28 14:19:33   0.169 sec              7.0            NaN   \n",
       "8     2022-10-28 14:19:33   0.177 sec              8.0            NaN   \n",
       "9     2022-10-28 14:19:33   0.185 sec              9.0            NaN   \n",
       "10    2022-10-28 14:19:33   0.194 sec             10.0            NaN   \n",
       "11    2022-10-28 14:19:33   0.202 sec             11.0            NaN   \n",
       "12    2022-10-28 14:19:33   0.213 sec             12.0            NaN   \n",
       "13    2022-10-28 14:19:33   0.224 sec             13.0            NaN   \n",
       "14    2022-10-28 14:19:33   0.234 sec             14.0            NaN   \n",
       "15    2022-10-28 14:19:33   0.243 sec             15.0            NaN   \n",
       "16    2022-10-28 14:19:33   0.251 sec             16.0            NaN   \n",
       "17    2022-10-28 14:19:33   0.261 sec             17.0            NaN   \n",
       "18    2022-10-28 14:19:33   0.270 sec             18.0            NaN   \n",
       "19    2022-10-28 14:19:33   0.279 sec             19.0            NaN   \n",
       "20    2022-10-28 14:19:33   0.291 sec             20.0            NaN   \n",
       "21    2022-10-28 14:19:33   0.301 sec             21.0            NaN   \n",
       "22    2022-10-28 14:19:33   0.310 sec             22.0       0.186434   \n",
       "\n",
       "    training_logloss  training_auc  training_pr_auc  training_lift  \\\n",
       "0                NaN           NaN              NaN            NaN   \n",
       "1                NaN           NaN              NaN            NaN   \n",
       "2                NaN           NaN              NaN            NaN   \n",
       "3                NaN           NaN              NaN            NaN   \n",
       "4                NaN           NaN              NaN            NaN   \n",
       "5                NaN           NaN              NaN            NaN   \n",
       "6                NaN           NaN              NaN            NaN   \n",
       "7                NaN           NaN              NaN            NaN   \n",
       "8                NaN           NaN              NaN            NaN   \n",
       "9                NaN           NaN              NaN            NaN   \n",
       "10               NaN           NaN              NaN            NaN   \n",
       "11               NaN           NaN              NaN            NaN   \n",
       "12               NaN           NaN              NaN            NaN   \n",
       "13               NaN           NaN              NaN            NaN   \n",
       "14               NaN           NaN              NaN            NaN   \n",
       "15               NaN           NaN              NaN            NaN   \n",
       "16               NaN           NaN              NaN            NaN   \n",
       "17               NaN           NaN              NaN            NaN   \n",
       "18               NaN           NaN              NaN            NaN   \n",
       "19               NaN           NaN              NaN            NaN   \n",
       "20               NaN           NaN              NaN            NaN   \n",
       "21               NaN           NaN              NaN            NaN   \n",
       "22          0.129014      0.989334         0.988704       1.990921   \n",
       "\n",
       "    training_classification_error  training_custom  validation_rmse  \\\n",
       "0                             NaN         0.693147              NaN   \n",
       "1                             NaN         0.508456              NaN   \n",
       "2                             NaN         0.403205              NaN   \n",
       "3                             NaN         0.333565              NaN   \n",
       "4                             NaN         0.286705              NaN   \n",
       "5                             NaN         0.254899              NaN   \n",
       "6                             NaN         0.232886              NaN   \n",
       "7                             NaN         0.217304              NaN   \n",
       "8                             NaN         0.200464              NaN   \n",
       "9                             NaN         0.190888              NaN   \n",
       "10                            NaN         0.182967              NaN   \n",
       "11                            NaN         0.173463              NaN   \n",
       "12                            NaN         0.167261              NaN   \n",
       "13                            NaN         0.161882              NaN   \n",
       "14                            NaN         0.158723              NaN   \n",
       "15                            NaN         0.155611              NaN   \n",
       "16                            NaN         0.151794              NaN   \n",
       "17                            NaN         0.147082              NaN   \n",
       "18                            NaN         0.140969              NaN   \n",
       "19                            NaN         0.139175              NaN   \n",
       "20                            NaN         0.136121              NaN   \n",
       "21                            NaN         0.131936              NaN   \n",
       "22                       0.043813         0.129014         0.196469   \n",
       "\n",
       "    validation_logloss  validation_auc  validation_pr_auc  validation_lift  \\\n",
       "0                  NaN             NaN                NaN              NaN   \n",
       "1                  NaN             NaN                NaN              NaN   \n",
       "2                  NaN             NaN                NaN              NaN   \n",
       "3                  NaN             NaN                NaN              NaN   \n",
       "4                  NaN             NaN                NaN              NaN   \n",
       "5                  NaN             NaN                NaN              NaN   \n",
       "6                  NaN             NaN                NaN              NaN   \n",
       "7                  NaN             NaN                NaN              NaN   \n",
       "8                  NaN             NaN                NaN              NaN   \n",
       "9                  NaN             NaN                NaN              NaN   \n",
       "10                 NaN             NaN                NaN              NaN   \n",
       "11                 NaN             NaN                NaN              NaN   \n",
       "12                 NaN             NaN                NaN              NaN   \n",
       "13                 NaN             NaN                NaN              NaN   \n",
       "14                 NaN             NaN                NaN              NaN   \n",
       "15                 NaN             NaN                NaN              NaN   \n",
       "16                 NaN             NaN                NaN              NaN   \n",
       "17                 NaN             NaN                NaN              NaN   \n",
       "18                 NaN             NaN                NaN              NaN   \n",
       "19                 NaN             NaN                NaN              NaN   \n",
       "20                 NaN             NaN                NaN              NaN   \n",
       "21                 NaN             NaN                NaN              NaN   \n",
       "22            0.142642        0.986037           0.984178          2.02988   \n",
       "\n",
       "    validation_classification_error  validation_custom  \n",
       "0                               NaN           0.693147  \n",
       "1                               NaN           0.509577  \n",
       "2                               NaN           0.404315  \n",
       "3                               NaN           0.335569  \n",
       "4                               NaN           0.289430  \n",
       "5                               NaN           0.257526  \n",
       "6                               NaN           0.235729  \n",
       "7                               NaN           0.220206  \n",
       "8                               NaN           0.204054  \n",
       "9                               NaN           0.195105  \n",
       "10                              NaN           0.187705  \n",
       "11                              NaN           0.178748  \n",
       "12                              NaN           0.172957  \n",
       "13                              NaN           0.168298  \n",
       "14                              NaN           0.165693  \n",
       "15                              NaN           0.163391  \n",
       "16                              NaN           0.160778  \n",
       "17                              NaN           0.156551  \n",
       "18                              NaN           0.151666  \n",
       "19                              NaN           0.150583  \n",
       "20                              NaN           0.148415  \n",
       "21                              NaN           0.144895  \n",
       "22                            0.049           0.142642  "
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_eval_only.scoring_history()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "037ab44c",
   "metadata": {},
   "source": [
    "Models `model_eval` and `model_eval_only` are guaranteed to be identical in behavior (same trees, same thresholds,..). The only technical difference between them is that the first one doesn't have full scoring history.\n",
    "\n",
    "We can also see that flag `score_eval_metric_only=True` saved us some training time. Model `model_eval_only` was built faster:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "b74d7bad",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Duration (s) with H2O scoring: 21.281\n",
      "Duration (s) with only eval_metric scored: 4.67\n"
     ]
    }
   ],
   "source": [
    "def total_duration(scoring_history):\n",
    "    return(sum(map(lambda x: float(x.strip().split(' ')[0]), scoring_history[\"duration\"].tolist())))\n",
    "\n",
    "\n",
    "print(\"Duration (s) with H2O scoring: %s\" % total_duration(model_eval.scoring_history()))\n",
    "print(\"Duration (s) with only eval_metric scored: %s\" % total_duration(model_eval_only.scoring_history()))\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "5935b7cb",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      0.025 sec\n",
       "1      0.139 sec\n",
       "2      0.179 sec\n",
       "3      0.228 sec\n",
       "4      0.280 sec\n",
       "5      0.356 sec\n",
       "6      0.434 sec\n",
       "7      0.511 sec\n",
       "8      0.587 sec\n",
       "9      0.676 sec\n",
       "10     0.773 sec\n",
       "11     0.854 sec\n",
       "12     0.943 sec\n",
       "13     1.058 sec\n",
       "14     1.160 sec\n",
       "15     1.246 sec\n",
       "16     1.345 sec\n",
       "17     1.462 sec\n",
       "18     1.566 sec\n",
       "19     1.684 sec\n",
       "20     1.802 sec\n",
       "21     1.916 sec\n",
       "22     2.057 sec\n",
       "Name: duration, dtype: object"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_eval.scoring_history()[\"duration\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b7b5211d",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0      0.030 sec\n",
       "1      0.118 sec\n",
       "2      0.127 sec\n",
       "3      0.135 sec\n",
       "4      0.144 sec\n",
       "5      0.151 sec\n",
       "6      0.161 sec\n",
       "7      0.169 sec\n",
       "8      0.177 sec\n",
       "9      0.185 sec\n",
       "10     0.194 sec\n",
       "11     0.202 sec\n",
       "12     0.213 sec\n",
       "13     0.224 sec\n",
       "14     0.234 sec\n",
       "15     0.243 sec\n",
       "16     0.251 sec\n",
       "17     0.261 sec\n",
       "18     0.270 sec\n",
       "19     0.279 sec\n",
       "20     0.291 sec\n",
       "21     0.301 sec\n",
       "22     0.310 sec\n",
       "Name: duration, dtype: object"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model_eval_only.scoring_history()[\"duration\"]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.6"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}
